Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions docs/adr/0011-interaction-guarantee-contract.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ trade-offs independently:
| --- | --- | --- |
| `runtime-selector` | daemon tree capture → `resolveSelectorChain` → guards → coordinate tap | full semantics |
| `runtime-ref` | session snapshot → ref lookup → guards → coordinate tap | full semantics |
| `target-drag` | independently resolve source + destination → guards → one pointer plan | dual-endpoint drag semantics |
| `direct-ios-selector` | selector sent to the XCTest runner, which queries and taps natively | saves a full snapshot round trip |
| `native-ref` | `backend.tapTarget`/`fillTarget` for `click @ref` / `fill @ref` | saves resolution round trips |
| `coordinate` | raw x/y tap | escape hatch; semantics intentionally minimal |
Expand Down Expand Up @@ -266,6 +267,10 @@ dimensions as their only frame source; cheaper frame sources such as
intentionally excluded because they can diverge from full-screen screenshot
coordinates on affected simulators.

Target-authored `gesture drag` is classified as the `target-drag` path above because it resolves two
elements and therefore inherits the element guarantees and ADR 0012 identity requirements for both.
Coordinate-authored drag remains outside this matrix.

Two-contact pan/pinch/rotate/transform planning is now owned by the typed
gesture-plan contract in [ADR 0013](0013-unified-gesture-plans.md). It remains a
sibling of this element-targeting matrix: coordinate gestures do not acquire
Expand Down
9 changes: 9 additions & 0 deletions docs/adr/0012-interactive-replay.md
Original file line number Diff line number Diff line change
Expand Up @@ -132,6 +132,15 @@ the corresponding provider contract cases in the same change.

### 3. Versioned `.ad` target-binding evidence

> **Amendment (#1567): dual-endpoint target evidence.** An action that resolves two elements records
> one `# agent-device:targets-v1 {"source":{...},"destination":{...}}` annotation instead of a
> single `target-v1`. Each nested value is a complete `TargetAnnotationV1` under the same identity,
> normalization, field-size, and verification rules. The wrapper is capped at 8,320 bytes. Replay
> verifies both selectors and carries two post-resolution guards before dispatch; either endpoint's
> selector miss, identity mismatch, unverifiable evidence, or dispatch-time guard mismatch refuses
> the entire action before pointer-down. Unknown future `targets-vN` annotations remain ordinary
> comments to an older reader, matching the single-target versioning rule.

Recording writes evidence for every action that resolves an element target. The plain-text format is a
versioned comment immediately before the action it annotates:

Expand Down
21 changes: 18 additions & 3 deletions docs/adr/0013-unified-gesture-plans.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,8 @@ while still routing through the canonical `pan` input.
The runtime plans canonical intent in `packages/contracts/src/gesture-plan.ts`. Contact topology is separate
from motion:

- one contact: pan or fling with a complete pointer trajectory and an explicit execution profile;
- one contact: pan, fling, or target-authored drag with a complete pointer trajectory and an explicit
execution profile;
- two contacts: pan, pinch, rotate, or transform with two complete, synchronized trajectories.

`swipe` without a duration remains public sugar for a fixed-duration fling. Timed public forms
Expand Down Expand Up @@ -133,8 +134,22 @@ Public two-finger pan is additive: `pointerCount?: 1 | 2` on pan and CLI
`kind`, `durationMs`, `pointerCount`, `from`, and `to` fields, followed by backend evidence.
Recording/replay keeps its existing public command identity and session semantics.

ADR 0011's element dispatch-path matrix remains unchanged: coordinate gestures do not resolve
selectors or refs and therefore cannot claim element-targeting guarantees.
Target-authored drag is additive: `gesture drag <source> <destination>` accepts a selector or pinned
snapshot ref at each endpoint, resolves both endpoints before device injection, then lowers their center
points to one uninterrupted single-pointer plan. The plan holds at the source, moves over the authored
duration, optionally holds at the destination, and releases. Both endpoint-resolution disclosures and
portable selector chains are returned to the caller. Recordings replace session-local refs at both
endpoints with those selector chains and attach one `targets-v1` annotation containing independent source
and destination identity evidence. Replay verifies both identities before pointer-down and guards both
independent endpoint resolutions against the verified elements. Ref admission happens for both endpoints
before either is dispatched, and the usual mutation boundary expires the frame after the gesture. Because
this contract includes the source hold, timed movement, and destination hold phases, target-authored drag
is admitted only on Android touch devices and iOS/iPadOS, whose adapters preserve the full plan.

ADR 0011's coordinate path remains unchanged: coordinate-authored gestures do not resolve selectors or
refs and therefore cannot claim element-targeting guarantees. Target-authored drag explicitly runs the
shared selector/ref resolution preflight before it enters the coordinate gesture executor; its endpoint
resolution and recording contracts are tested at that composition seam.

## Consequences

Expand Down
15 changes: 15 additions & 0 deletions examples/test-app/replays/drag-android.ad
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
context platform=android kind=emulator timeout=60000

env APP_TARGET="com.callstack.agentdevicelab"
env APP_URL=""

open "${APP_TARGET}" --relaunch --launch-url "${APP_URL}"
react-native dismiss-overlay
wait "Gesture lab" 30000
wait "gesture canary ready" 5000
wait "drag completed no" 5000

gesture drag 'id="drag-source"' 'id="drag-destination"' 600 500 0
wait "drag completed yes" 5000

close
14 changes: 14 additions & 0 deletions examples/test-app/replays/drag.ad
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
context platform=ios kind=simulator timeout=60000

env APP_TARGET="Agent Device Tester"
env APP_URL=""

open "${APP_TARGET}" --relaunch --launch-url "${APP_URL}"
wait "Gesture lab" 30000
wait "gesture canary ready" 5000
wait "drag completed no" 5000

gesture drag 'id="drag-source"' 'id="drag-destination"' 600 500 0
wait "drag completed yes" 5000

close
34 changes: 34 additions & 0 deletions examples/test-app/src/screens/GestureLab.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,7 @@ export function GestureLab() {
const [canaryReady, setCanaryReady] = useState(false);
const [transform, setTransform] = useState<TransformState>(initialTransform);
const [counts, setCounts] = useState<GestureCounts>(initialCounts);
const [dragCompleted, setDragCompleted] = useState(false);
const transformRef = useRef<TransformState>(initialTransform);
const gestureStartRef = useRef<TransformState>(initialTransform);
const androidTouchStartRef = useRef<AndroidTouchStart | undefined>(undefined);
Expand Down Expand Up @@ -191,6 +192,15 @@ export function GestureLab() {
.minDistance(4)
.runOnJS(true)
.onStart(handleTwoPointerPan);
const holdDragGesture = Gesture.Pan()
.activateAfterLongPress(500)
.minDistance(10)
.runOnJS(true)
.onEnd((event, completed) => {
if (completed && Math.hypot(event.translationX, event.translationY) > 60) {
setDragCompleted(true);
}
});
const legacyFlingRefs = [
legacyFlingLeftRef,
legacyFlingRightRef,
Expand Down Expand Up @@ -337,6 +347,30 @@ export function GestureLab() {
{changeStatusLabel}
</Text>
</View>

<View style={styles.dragRow} testID="drag-gesture-fixture">
<GestureDetector gesture={holdDragGesture}>
<View
accessible
accessibilityLabel="Drag source"
style={styles.dragEndpoint}
testID="drag-source"
>
<Text style={styles.dragEndpointLabel}>Source</Text>
</View>
</GestureDetector>
<View
accessible
accessibilityLabel="Drag destination"
style={styles.dragEndpoint}
testID="drag-destination"
>
<Text style={styles.dragEndpointLabel}>Destination</Text>
</View>
</View>
<Text style={styles.metric} testID="drag-gesture-status">
drag completed {dragCompleted ? 'yes' : 'no'}
</Text>
</SectionCard>
);
}
Expand Down
20 changes: 20 additions & 0 deletions examples/test-app/src/screens/gesture-lab-styles.ts
Original file line number Diff line number Diff line change
Expand Up @@ -44,5 +44,25 @@ export function createGestureLabStyles(colors: AppColors) {
fontWeight: '600',
lineHeight: 18,
},
dragRow: {
flexDirection: 'row',
gap: 20,
justifyContent: 'space-between',
},
dragEndpoint: {
alignItems: 'center',
backgroundColor: colors.cardStrong,
borderColor: colors.line,
borderRadius: 8,
borderWidth: 1,
flex: 1,
height: 56,
justifyContent: 'center',
},
dragEndpointLabel: {
color: colors.text,
fontSize: 13,
fontWeight: '700',
},
});
}
26 changes: 26 additions & 0 deletions packages/ad-replay/src/internal/__tests__/plan-digest.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -118,6 +118,32 @@ test('computeReplayPlanDigest changes when target evidence consumed before actio
}
});

test('computeReplayPlanDigest binds both entries of multi-target evidence', () => {
const evidence = {
role: 'view',
ancestry: [],
sibling: 0,
viewportOrder: 0,
verification: 'verified' as const,
};
const targetEvidences = {
source: { ...evidence, id: 'source' },
destination: { ...evidence, id: 'destination' },
};
const original = digestFor([action({ targetEvidences })]);
assert.notEqual(
original,
digestFor([
action({
targetEvidences: {
...targetEvidences,
destination: { ...targetEvidences.destination, id: 'different-destination' },
},
}),
]),
);
});

test('computeReplayPlanDigest never changes based on unsubstituted ${VAR} text (variable VALUES never affect the digest)', () => {
// The digest is computed over the still-unsubstituted action text; --env
// values are resolved later, at invocation time, so two runs with
Expand Down
1 change: 1 addition & 0 deletions packages/ad-replay/src/internal/plan-digest.ts
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,7 @@ function canonicalizeAction(
flags: action.flags ?? {},
runtime: action.runtime ?? null,
targetEvidence: action.targetEvidence ?? null,
targetEvidences: action.targetEvidences ?? null,
source: { path: sourcePath, line },
};
}
8 changes: 8 additions & 0 deletions packages/ad-replay/src/internal/runtime-port-types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -150,6 +150,13 @@ export type AdReplayTargetBindingEvidence = Readonly<{
/** The pre-action guard `dispatchStep` threads to the interaction layer's own resolution. */
export type AdReplayDispatchGuard = Readonly<
| { readonly kind: 'target'; readonly guard: AdReplayVerifiedTargetGuard }
| {
readonly kind: 'targets';
readonly guards: Readonly<{
source: AdReplayVerifiedTargetGuard;
destination: AdReplayVerifiedTargetGuard;
}>;
}
| { readonly kind: 'landmark'; readonly landmark: TargetAnnotationV1 }
>;

Expand Down Expand Up @@ -210,6 +217,7 @@ export type AdReplayStepRuntime = Readonly<{
action: SessionAction,
resolvedAction: SessionAction,
index: number,
targetRole?: 'source' | 'destination',
): AdReplayVerificationEntry;
/**
* Captures a fresh snapshot for classification or for a divergence's
Expand Down
1 change: 1 addition & 0 deletions packages/ad-replay/src/internal/target-verification.ts
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,7 @@ export type AdReplayTargetStructuralDenotation = Readonly<{
* sees the untyped bag itself.
*/
export type AdReplayGuardMismatchEvidence = Readonly<{
targetRole?: 'source' | 'destination';
observed: LocalIdentity | undefined;
expectedStructural: AdReplayTargetStructuralDenotation | undefined;
observedStructural: AdReplayTargetStructuralDenotation | undefined;
Expand Down
Loading
Loading