Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .github/workflows/sync.yml
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,9 @@ jobs:
# The curated Concepts and Ecosystem sections replace it. Re-enable only
# if a maintained, product-accurate architecture page is reintroduced.

- name: Validate published version claims
run: uv run python scripts/check_published_version_claims.py --offline

- name: Run tests against generated source
run: uv run pytest tests/ -q

Expand Down
13 changes: 13 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,19 @@ uv run mkdocs build --strict
uv run python scripts/validate_docs.py
```

The managed-runtime artifact tuple has one editable source:
`docs/published-version-claims.json`. Change the claim's `version`, release
commit, archive hashes, source-manifest evidence, and `verified_on` fields once,
then render all marked pages:

```bash
uv run python scripts/render_published_version_claims.py
uv run python scripts/render_published_version_claims.py --check
```

Do not edit the marked page values directly. The offline version-claim gate and
the documentation deployment both fail when the committed render is stale.

## Tests

```bash
Expand Down
18 changes: 16 additions & 2 deletions docs/concepts/backends.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,19 +28,33 @@ the highest-fidelity signal each surface offers.

### Web: Playwright

A headless Chromium driven by Playwright drives the web substrate, and every
example in these docs uses it. It exposes a full structured layer:
Playwright drives the web substrate. Flow can launch Chromium or attach to one
existing signed-in local Chromium tab through a loopback CDP endpoint. Both
entry modes use the same recorder, compiler format, and governed runtime. The
browser exposes a full structured layer:

- **Structural rung**: reads the DOM element under a point, so resolution and
[identity](identity-gate.md) can use stable selectors and structured text
where they exist.
- **Structural postconditions**: URL change, title change, new-tab opened.
- **CI-friendly**: no OS permissions, no display server; the whole record →
compile → replay loop runs in a container.
- **Existing-session recording**: attach mode preserves a dedicated browser
profile that has already completed sign-in, SSO, or 2FA. Flow refuses remote
endpoints and ambiguous same-origin tabs. It does not navigate or close the
external browser. It records viewport and monitor-scale transitions as new
per-event coordinate baselines. An idle transition rebaselines and continues.
An action that overlaps an unverified transition aborts the recording and
publishes no complete metadata.

It shares the same bundle, resolution ladder, and identity gate as every other
substrate; nothing about the safety model is specific to it.

The custom Chrome extension in `openadapt-capture` remains a development
prototype. Browser acquisition stays Playwright-native because source-time
secret exclusion, DOM identity, field geometry, and exact event/frame binding
are part of the supported compiler contract.

### Desktop: Windows (UIA)

The public `WindowsBackend` now narrows the in-session boundary to typed
Expand Down
8 changes: 4 additions & 4 deletions docs/concepts/deployment-matrix.md
Original file line number Diff line number Diff line change
Expand Up @@ -108,11 +108,11 @@ synthetic. A missing production dependency makes the affected operation
unavailable rather than substituting a simulated success.

The retained hosted-recorder qualification used a Flow 1.8.0 worker. The
current Cloud managed-runtime manifest pins Flow 1.31.0 at release commit
`2d225dea9a0ad29ca84ce1b037cc0ac671367e28`. Its wheel SHA-256 is
`81133db1528ad1bb1f26e3fcb6aea61b0651db6d905cf2e4943e8383c1f3d29c` and
current Cloud managed-runtime manifest pins Flow <!-- version-claim:hosted-runner-managed-runtime-pin:version -->1.31.0<!-- /version-claim:hosted-runner-managed-runtime-pin:version --> at release commit
<code><!-- version-claim:hosted-runner-managed-runtime-pin:release_commit -->2d225dea9a0ad29ca84ce1b037cc0ac671367e28<!-- /version-claim:hosted-runner-managed-runtime-pin:release_commit --></code>. Its wheel SHA-256 is
<code><!-- version-claim:hosted-runner-managed-runtime-pin:wheel_sha256 -->81133db1528ad1bb1f26e3fcb6aea61b0651db6d905cf2e4943e8383c1f3d29c<!-- /version-claim:hosted-runner-managed-runtime-pin:wheel_sha256 --></code> and
its source SHA-256 is
`cf1fc356d14d267df82be188de3e9a3575734f18f46ef91ac8075438cc731540`.
<code><!-- version-claim:hosted-runner-managed-runtime-pin:sdist_sha256 -->cf1fc356d14d267df82be188de3e9a3575734f18f46ef91ac8075438cc731540<!-- /version-claim:hosted-runner-managed-runtime-pin:sdist_sha256 --></code>.
The pin proves configured artifact identity. It does not prove that the build is
deployed or that a hosted workflow passed acceptance. Public readiness checks
live mode, authentication, database, private storage, runner, compiler,
Expand Down
4 changes: 2 additions & 2 deletions docs/ecosystem/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,8 +34,8 @@ module inside it:
| [openadapt-flow](https://github.com/OpenAdaptAI/openadapt-flow) | **Beta** | Canonical compiler and governed runtime. Drives web, native Windows, native macOS, native Linux, RDP, and Citrix/VDI as first-class substrates behind one backend protocol. |
| [OpenAdapt Cloud](https://app.openadapt.ai/) | **Beta** | Proprietary live control plane for the managed subscription: organizations, exact-hash admission, runner orchestration, reports, billing, and usage. |
| [openadapt-desktop](https://github.com/OpenAdaptAI/openadapt-desktop) | **Beta** | Public `desktop-v0.15.0` provides Windows MSI/NSIS, macOS arm64/x64 DMG, and Linux AppImage/DEB installers. Every installer path is installed, launched, and uninstalled in the native release workflow; the release includes exact checksums, a CycloneDX SBOM, platform metadata, and build-provenance attestations. |
| [openadapt-agent](https://github.com/OpenAdaptAI/openadapt-agent) | **Experimental** | Active v2 bridge that exposes governed Flow bundles to MCP clients and Agent Skills. The pre-v2 model-driven execution wrapper is the deprecated line; the repository itself is active. |
| [openadapt-capture](https://github.com/OpenAdaptAI/openadapt-capture) | **Beta** | Canonical native screen, mouse, keyboard, timing, and window-scoped recorder behind Flow's Windows, macOS, Linux, RDP, and Citrix recording paths. Capture 1.1 retains Windows UIA evidence at action time; remote sessions remain externally black-box. Browser recording remains inside Flow's Playwright listener. |
| [openadapt-agent](https://github.com/OpenAdaptAI/openadapt-agent) | **Beta** | Active v2 bridge that exposes governed Flow bundles to MCP clients and Agent Skills. The pre-v2 model-driven execution wrapper is the deprecated line; the repository itself is active. |
| [openadapt-capture](https://github.com/OpenAdaptAI/openadapt-capture) | **Beta** | Canonical native screen, mouse, keyboard, timing, and window-scoped recorder behind Flow's Windows, macOS, Linux, RDP, and Citrix recording paths. Capture 1.2 retains Windows UIA evidence at action time; remote sessions remain externally black-box. Browser recording remains inside Flow's Playwright recorder, which can launch Chromium or attach to one existing local tab. The custom Capture extension remains a prototype. |
| [openadapt-privacy](https://github.com/OpenAdaptAI/openadapt-privacy) | **Experimental** | Optional PII/PHI scrubbing used on configured persist, log, and upload paths. |
| [openadapt-types](https://github.com/OpenAdaptAI/openadapt-types) | **Experimental** | Shared interoperability schemas; contributor-facing, not an end-user product. |

Expand Down
6 changes: 6 additions & 0 deletions docs/get-started/first-workflow.md
Original file line number Diff line number Diff line change
Expand Up @@ -108,6 +108,12 @@ profiles require it explicitly.)
Perform the task once. When you are done, press ++ctrl+c++ or close the browser
window to finish. The recording is written to `rec/`.

!!! tip "Keep an existing SSO or 2FA session"
Start a dedicated Chromium profile with a local remote-debugging port. Add
`--browser-cdp-endpoint http://127.0.0.1:9222` to attach the same recorder
to its one open same-origin tab. Flow does not navigate or close that
browser. See [Record your own app](../guides/record-your-app.md#use-an-existing-signed-in-chromium-session).

!!! tip "Record a clean demonstration"
Do the task the way you want it replayed: one clear path, no dead ends. The
compiler treats your demonstration as evidence of intent, so a tidy run
Expand Down
4 changes: 2 additions & 2 deletions docs/get-started/what-works-today.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ customer-controlled runtime connected to the same governance model.
| Surface | Status | What is actually demonstrated | Important boundary |
|---|---|---|---|
| `openadapt` installer and `openadapt flow` dispatcher | **Supported** | Installs the compiler and exposes the unified command surface. | The standalone `openadapt-flow` package remains the canonical engine and may be installed directly. |
| Record -> compile -> lint -> certify -> replay -> report on a browser | **Supported** | Runs end to end in CI against the bundled app and end to end against a real third-party app. | A clean run is not automatically safe. Identity coverage, risk classification, postconditions, and effect contracts must be audited per bundle. |
| Record -> compile -> lint -> certify -> replay -> report on a browser | **Supported** | Runs end to end in CI against the bundled app and end to end against a real third-party app. The local attach recorder also passes 3 real Chromium record-and-compile trials with source-time password exclusion and external-browser survival checks. | A clean run is not automatically safe. Identity coverage, risk classification, postconditions, and effect contracts must be audited per bundle. Attach mode is loopback-only Chromium and requires a dedicated browser process started with remote debugging. The custom Capture extension remains a prototype. |
| Deterministic target re-resolution and saved heal diffs | **Supported** | Theme, movement, and rename drift are covered by the bundled drift matrix. | Scale/reflow and tenant-specific state can still halt. The base bundle is not silently promoted; save and review a healed bundle explicitly. |
| `lint` and `certify` | **Supported** | Report coverage gaps and refuse bundles that violate the selected policy. | Certification is opt-in and only enforces what the policy names. An uncertified bundle remains runnable with `replay`. |
| Fail-closed `run` admission gate | **Supported** | The shipped gate checks certification, identity/effect coverage, approval fallback, encryption, and manifest integrity before executing. | Development escape hatches exist, and passing the gate does not validate the backend or prove a workflow safe. |
Expand All @@ -72,7 +72,7 @@ customer-controlled runtime connected to the same governance model.
| Hosted CLI connectivity | **Supported / public offer** | `login`, exact-hash artifact preparation/upload, one-time runtime validation, bound replacement activation, and `report-break` connect the local engine to the live control plane. | Upload requires destination policy and an approved sanitized derivative; checkout never bypasses an egress refusal. |
| Artifact sanitation and local review | **Supported / launch gate** | The sanitized-derivative pipeline inventories, transforms, rescans, manifests, hashes, and supports local review/approval. | The raw original remains sensitive; unknown or unresolved content is refused; runtime observations can reintroduce PHI/PII. |
| Cross-engine hosted validation | **Supported / launch gate** | `validate-hosted` binds an approved recording and bundle, compiler provenance, strict lint, policy certification, derived risk class, and successful replay report to a one-time Cloud challenge. | It is operator self-attestation signed with the ingest token, not an independently observed certification. Exact deployment policy, risk-class, and deployed compiler-version allowlists still apply. |
| Hosted browser recorder and runtime health | **Supported / bounded launch component** | A retained non-simulated hosted session on `openadapt-flow` 1.8.0 produced frames and input evidence, assembled a compileable recording, finalized one workflow idempotently, enforced resource limits, and cleaned up ephemeral qualification data. The current managed-runtime manifest pins the Flow 1.31.0 runner/compiler artifact identity. Authenticated live health separately checks live mode, auth, database, storage, runner, compiler, runtime-validation trust, runtime boundary, bundle protection, recorder, callbacks, scheduler, human-decision Web Push, retention, security events, secrets, validation policy, and billing. | Explicitly initiated, public-HTTPS, non-regulated authoring only. Raw observations remain private inside the declared hosted boundary. A runtime pin does not prove live deployment or hosted acceptance. Readiness proves deployed dependencies and service identity, not a customer workflow qualification or SLA. |
| Hosted browser recorder and runtime health | **Supported / bounded launch component** | A retained non-simulated hosted session on `openadapt-flow` 1.8.0 produced frames and input evidence, assembled a compileable recording, finalized one workflow idempotently, enforced resource limits, and cleaned up ephemeral qualification data. The current managed-runtime manifest pins the Flow <!-- version-claim:hosted-runner-managed-runtime-pin:version -->1.31.0<!-- /version-claim:hosted-runner-managed-runtime-pin:version --> runner/compiler artifact identity. Authenticated live health separately checks live mode, auth, database, storage, runner, compiler, runtime-validation trust, runtime boundary, bundle protection, recorder, callbacks, scheduler, human-decision Web Push, retention, security events, secrets, validation policy, and billing. | Explicitly initiated, public-HTTPS, non-regulated authoring only. Raw observations remain private inside the declared hosted boundary. A runtime pin does not prove live deployment or hosted acceptance. Readiness proves deployed dependencies and service identity, not a customer workflow qualification or SLA. |
| Hosted dashboard/control plane | **Supported / public offer** | Authentication, organizations, exact-hash bundle ingest, immutable run admission, browser runner orchestration, structural reports, replacement activation, billing, and metering form the managed lifecycle. | Production uses live dependencies and fails unavailable rather than substituting mock behavior. |
| Hosted execution | **Supported / public offer** | Live Stripe Checkout connects onboarding and subscription entitlements to managed browser execution; the runner verifies exact admitted bundle bytes and authenticated callbacks. | The public subscription covers approved browser workflows. Other substrates use separately scoped deployments and commercial terms. Checkout does not create an SLA or certification. |
| Air-gapped on-prem package | **Supported** | A local queue, systemd unit, minimized hash-chained audit log, and air-gap checks are provided. | Full-disk encryption and operational hardening remain operator/deployment responsibilities. |
Expand Down
4 changes: 2 additions & 2 deletions docs/guides/hosted.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ switching, and sign-out.
| Surface | Launch status | Boundary |
|---|---|---|
| Local browser record -> compile -> managed execute | **Beta / public offer** | Governed authoring and validation remain local; managed execution uses the qualified browser substrate. |
| Hosted browser record -> compileable workflow | **Beta / bounded launch component** | The retained non-simulated provider qualification used `openadapt-flow` 1.8.0; the current managed-runtime manifest pins Flow 1.31.0 artifact identity. A runtime pin does not prove live deployment or hosted workflow acceptance. This is a separate raw-observation boundary, not the reviewed-derivative upload lane. |
| Hosted browser record -> compileable workflow | **Beta / bounded launch component** | The retained non-simulated provider qualification used `openadapt-flow` 1.8.0; the current managed-runtime manifest pins Flow <!-- version-claim:hosted-runner-managed-runtime-pin:version -->1.31.0<!-- /version-claim:hosted-runner-managed-runtime-pin:version --> artifact identity. A runtime pin does not prove live deployment or hosted workflow acceptance. This is a separate raw-observation boundary, not the reviewed-derivative upload lane. |
| Account, organization, onboarding | **Beta / public offer** | Checkout and sign-in bind the subscription to an isolated organization. |
| Structural run history and reports | **Beta / public offer** | Safety depends on the workflow's configured identity, effect, and policy checks. Repair and validation remain local. |
| Checkout, portal, entitlements, metering | **Beta / public offer** | Live Stripe Checkout, signed webhooks, entitlements, usage, and the billing portal form one managed subscription contract. |
Expand All @@ -64,7 +64,7 @@ demo. A qualified hosted browser session produced PNG frames, accepted and
retained input evidence, assembled a native recording, created one compileable
workflow idempotently, enforced its resource limits, and removed the ephemeral
qualification data. That retained qualification used an `openadapt-flow` 1.8.0
worker. The managed-runtime manifest pins the Flow 1.31.0 artifact identity. A
worker. The managed-runtime manifest pins the Flow <!-- version-claim:hosted-runner-managed-runtime-pin:version -->1.31.0<!-- /version-claim:hosted-runner-managed-runtime-pin:version --> artifact identity. A
pin does not prove that the build is live or that a hosted workflow passed
acceptance. The public readiness endpoint separately verifies the configured
live dependencies, including authentication, storage, runner, compiler,
Expand Down
55 changes: 52 additions & 3 deletions docs/guides/record-your-app.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,9 @@ target flags in place of `--url`.

## Record

On the web substrate, `record --backend web --url` opens a headed browser on your app and
watches what you do: clicks, typing, key presses, and scrolls. It writes the same
recording format `compile` consumes.
On the web substrate, `record --backend web --url` opens a headed browser on
your app and watches what you do: clicks, typing, key presses, and scrolls. It
writes the same recording format `compile` consumes.

```bash
openadapt flow record --backend web --url https://your.app --out rec
Expand All @@ -23,6 +23,55 @@ the window to finish.
Add `--headless` to run the browser without a window, for scripted recording
in a pipeline.

### Use an existing signed-in Chromium session

Flow can attach the same Playwright recorder to one existing local Chromium
tab. Use this mode when the browser profile has already completed sign-in, SSO,
or 2FA.

Start Chromium with a dedicated debugging profile. For example, on macOS:

```bash
"/Applications/Google Chrome.app/Contents/MacOS/Google Chrome" \
--remote-debugging-address=127.0.0.1 \
--remote-debugging-port=9222 \
--user-data-dir="./.openadapt-chrome-profile"
```

Open the application in that browser. Then run:

```bash
openadapt flow record --backend web --url https://your.app \
--browser-cdp-endpoint http://127.0.0.1:9222 --out rec
```

Flow selects the sole open tab on the `--url` origin. It refuses an ambiguous
selection. If the browser has two or more tabs on that origin, add the exact
current URL with `--browser-page-url`. Flow does not navigate or close the
attached browser.

The endpoint must be on localhost or a loopback IP address. The attached
screenshots use the tab's actual CSS viewport, so retained frames and DOM input
coordinates stay aligned on high-density displays. Password fields and fields
declared with `--secret` keep the same source-time exclusion and frame-redaction
contract as launch mode.

Keep the selected tab on the declared application origin. You can resize the
tab or move its window between monitors while no action is in progress. Flow
observes viewport and device-scale changes. It waits for a stable CSS-pixel
frame and binds later events to the new coordinate space. The recording keeps
the viewport history and the exact before and after viewport for each event.

Flow refuses a cross-origin navigation or an event from an iframe. It also
refuses an action that overlaps a resize or monitor-scale transition. The last
refusal is necessary because no exact pre-action frame exists in the new
coordinate space. An overlapping action aborts the recording and publishes no
complete metadata. When you resize between actions, stop interacting until the
new frame is stable. Recording then continues automatically.

The custom Chrome extension in `openadapt-capture` is a development prototype.
It is not this supported path and it is not a governed replay mechanism.

## Compile and replay

```bash
Expand Down
Loading