Add Kernel Vault credentials to the Jev agent

packages/browser-loop/examples/jev-system-one/README.md

@@ -4,9 +4,10 @@ This example runs a custom browser-agent loop with TypeSafe AI's Jev. It does no
 
 The loop uses:
 
-- page-specific `CLICK`, `TYPE_TEXT`, `SELECT`, `SCROLL`, and `WAIT` candidates;
+- page-specific `CLICK`, `TYPE_TEXT`, `SELECT`, `USE_CREDENTIALS`, `SCROLL`, and `WAIT` candidates;
 - speculative operation and target questions in one System One request;
-- a small text-model escape hatch only after Jev selects a field or navigation operation;
+- logical credential-form extraction with value-redacted password and OTP state;
+- a small text-model escape hatch only after Jev selects a non-credential field or navigation operation;
 - code-owned target guards, step limits, and repeated-no-change detection;
 - `DONE` and `BLOCKED` as explicit Jev choices.
 
@@ -18,11 +19,13 @@ Navigation is part of the loop. The runner opens a blank tab before starting; st
 flowchart LR
   O[Browser observation] --> C[Build candidate space]
   C --> J[Jev operation and target]
-  J --> R{Needs text?}
-  R -->|no| L[Lower candidate]
-  R -->|yes| T[Text resolver]
+  J --> R{Candidate kind?}
+  R -->|ordinary| L[Lower candidate]
+  R -->|text| T[Text resolver]
+  R -->|credential form| V[Vault credential broker]
   T --> L
   L --> E[BrowserExecutor.execute]
+  V --> E
   E --> O
 ```
 
@@ -39,6 +42,7 @@ Examples:
 | --- | --- |
 | Click Search | target guard, then `browser_click` at its current viewport point |
 | Type in From | text resolver, guarded click, `CTRL+A`, then `browser_type` |
+| Use credentials for Sign in | Jev vault-item choice, optional HITL collection, then grouped Vault `fill` |
 | Select Business | guarded native-select update |
 | Navigate | `browser_navigate` |
 | Done / blocked | no browser action |
@@ -51,6 +55,7 @@ Requirements:
 - `KERNEL_API_KEY`
 - `TYPESAFE_API_KEY`
 - `TEXT_MODEL_API_KEY` for tasks that require navigation inference or text entry
+- optional `KERNEL_VAULT` to enable credential-form actions against that project-scoped vault
 
 The text helper uses an OpenAI-compatible `/chat/completions` endpoint. It limits responses to 1,024 tokens and retries once when a provider returns malformed JSON, before any browser mutation:
 
@@ -77,6 +82,17 @@ npm run run -- \
   --task "Open https://news.ycombinator.com, then open the newest submissions page using the new link"
 ```
 
+Set `KERNEL_VAULT` to an existing or new vault name to enable credential handling. The runner links that vault when it creates the browser, because browser-vault links are immutable. Jev receives every credential item object returned by the Vault API, whose contract omits sensitive values, plus the current URL, title, and extracted form. It may select an existing item, create one credential item for the current visible form, or decline a safe match.
+
+```bash
+export KERNEL_VAULT="browser-agent"
+npm run run -- --task "Open https://example.com/login and sign in"
+```
+
+When a new or incomplete item needs user input, the runner opens the time-limited Kernel collection form and waits for the item to become ready. The collection URL and field values are excluded from Jev history and progress logs. Password, OTP, and vault-mapped browser controls expose only `has_value` to Jev, never their raw DOM values.
+
+Run the opt-in live form detector against 15 public login pages and three negative controls with `npm run smoke:credentials`. It requires `KERNEL_API_KEY`, creates one temporary browser, does not submit any form, and fails on missed forms, false positives, exposed credential-field values, or per-field actions that bypass the grouped Vault action.
+
 ### Install into a browser REPL
 
 The browser process API and REPL share a filesystem. This CLI flow downloads the prebuilt agent module to `/tmp/browser-loop/jev-agent.mjs`, defines `runJev` once, and reuses it in a later REPL call:
@@ -133,6 +149,7 @@ Jev receives more than the candidate labels. Every question is conditioned on st
     "text": ""
   },
   "elements": [],
+  "credential_forms": [],
   "recent_actions": []
 }
 ```
@@ -145,7 +162,10 @@ The operation question contains only currently available operations. Target ques
 - `actions.ts`: page-specific candidate construction
 - `browser.ts`: `BrowserExecutor` adapter, target validation, and execution
 - `snapshot.ts`: viewport control and text observation
+- `credentials.ts`: visible credential-form extraction, redaction, and selector preparation
 - `models.ts`: Jev System One operation and target policy
+- `vault-models.ts`: Jev credential-item selection and field mapping
+- `vault.ts`: Kernel Vault collection, polling, and grouped fill
 - `text.ts`: optional OpenAI-compatible string resolver
 - `run.ts`: Kernel browser setup and CLI
 - `repl.ts`: persistent-REPL agent factory
@@ -157,4 +177,6 @@ The operation question contains only currently available operations. Target ques
 
 This is deliberately a custom example rather than a generalized policy API. Its observation pass includes only controls whose center is inside the current viewport, records each control's executable operations from its underlying DOM element, and assigns a stable identity for the life of the document. When a visible cross-origin frame is present, it supplements that state with the frame controls from Browser Loop's stitched accessibility observation. Before input, the runtime validates only the selected control's identity and state. A stale target causes a fresh observation and policy decision; snapshot-scoped references are not remapped.
 
-Editable controls expose separate `TYPE_TEXT` and `Open …` click candidates so Jev can distinguish entering a literal from opening an autocomplete or picker. The example does not generate prose answers, handle CAPTCHA, upload files, or enter passwords. The viewport candidate list is bounded to 250 grounded actions. Page text is treated as untrusted data, and the text resolver returns `null` when required information is absent.
+Editable controls expose separate `TYPE_TEXT` and `Open …` click candidates so Jev can distinguish entering a literal from opening an autocomplete or picker. When Vault support is enabled, fields in a detected credential form expose only its grouped `USE_CREDENTIALS` action; sign-in goals prioritize that action before ordinary page operations. Credential fields are grouped by their native form or nearest primary authentication action. Main-document credential fields can be filled through Kernel Vaults; cross-origin accessibility-only and shadow-DOM fields remain observable but do not receive vault candidates because the fill API requires document CSS selectors.
+
+The example does not generate prose answers, handle CAPTCHA, or upload files. The viewport candidate list is bounded to 250 grounded actions. Page text is treated as untrusted data, and the text resolver returns `null` when required information is absent. Vault fill failures and unknown outcomes are not automatically retried.

packages/browser-loop/examples/jev-system-one/accessibility.ts

@@ -21,6 +21,8 @@ export function elementsFromAccessibilitySnapshot(snapshot: string, visibleFrame
 		const line = lines[index]!;
 		if (!line.ref || consumedOptions.has(line.ref)) continue;
 		const states = elementState(line.states);
+		const credentialSemantic = credentialSemanticOf(line.role, line.name);
+		const sensitive = credentialSemantic === "password" || credentialSemantic === "otp";
 		let operations: ObservationElement["operations"] = [];
 		let options: ObservationElement["options"] = [];
 		if (line.role === "combobox") {
@@ -31,15 +33,16 @@ export function elementsFromAccessibilitySnapshot(snapshot: string, visibleFrame
 				for (const option of descendants) if (option.ref) consumedOptions.add(option.ref);
 			} else operations = ["CLICK"];
 		} else if (["textbox", "searchbox", "spinbutton"].includes(line.role)) {
-			operations = ["TYPE_TEXT", "CLICK"];
+			operations = sensitive ? ["CLICK"] : ["TYPE_TEXT", "CLICK"];
 		} else if (CLICKABLE_ROLES.has(line.role)) operations = ["CLICK"];
 		if (!operations.length) continue;
 		additions.push({
 			id: `ax:${line.ref}`,
 			node: -Number(line.ref.slice(1)),
 			role: line.role,
 			name: line.name || line.role,
-			value: states.value ?? "",
+			value: sensitive ? "" : states.value ?? "",
+			...(sensitive ? { hasValue: Boolean(states.value), credentialSemantic, sensitive: true } : {}),
 			operations,
 			options,
 			...(states.checked === undefined ? {} : { checked: states.checked }),
@@ -163,6 +166,13 @@ function elementState(states: ReadonlyMap<string, string | boolean | number>): {
 	};
 }
 
+function credentialSemanticOf(role: string, name: string): "password" | "otp" | undefined {
+	if (!["textbox", "searchbox", "spinbutton"].includes(role)) return undefined;
+	if (/password|passphrase|passcode/i.test(name)) return "password";
+	if (/\b(?:otp|one[ -]?time|verification|authenticator)\s*(?:code)?\b/i.test(name)) return "otp";
+	return undefined;
+}
+
 function quotedStringEnd(value: string): number {
 	let escaped = false;
 	for (let index = 1; index < value.length; index++) {

packages/browser-loop/examples/jev-system-one/actions.ts

@@ -4,9 +4,17 @@ const MAX_GROUNDED_CANDIDATES = 250;
 const SECRET_FIELD = /\b(?:password|passphrase)\b/i;
 const FILE_CONTROL = /\b(?:choose file|upload file)\b/i;
 
-export function buildCandidateSpace(observation: Observation, goal: string, history: readonly HistoryEntry[] = []): JevCandidateSpace {
+export function buildCandidateSpace(
+	observation: Observation,
+	goal: string,
+	history: readonly HistoryEntry[] = [],
+	options: { credentials?: boolean } = {},
+): JevCandidateSpace {
 	const candidates: JevCandidate[] = [];
 	const navigationOnly = observation.url === "about:blank" || observation.url.startsWith("chrome://");
+	const credentialNodes = options.credentials
+		? new Set(observation.credentialForms.flatMap((form) => form.fields.map((field) => field.target.node)))
+		: new Set<number>();
 	const pageElements = navigationOnly ? [] : observation.elements.filter((element) => !isExcludedControl(element));
 	const operationsByNode = new Map<number, Set<ElementOperation>>();
 	let grounded = 0;
@@ -24,7 +32,7 @@ export function buildCandidateSpace(observation: Observation, goal: string, hist
 	};
 
 	for (const element of pageElements) {
-		if (element.disabled) continue;
+		if (element.disabled || credentialNodes.has(element.node)) continue;
 		const target: ElementTarget = {
 			documentId: observation.documentId,
 			node: element.node,
@@ -39,9 +47,10 @@ export function buildCandidateSpace(observation: Observation, goal: string, hist
 						id: `select:${element.id}:${optionIndex + 1}`,
 						kind: "target",
 						operation,
-						label: `Select ${JSON.stringify(option.label)} in ${JSON.stringify(element.name)}; current value=${JSON.stringify(element.value)}`,
+						label: `Select ${JSON.stringify(option.label)} in ${JSON.stringify(element.name)}${stateDescription(element)}`,
 						target,
 						value: option.value,
+						hasValue: element.hasValue,
 					})) break;
 				}
 				continue;
@@ -51,9 +60,10 @@ export function buildCandidateSpace(observation: Observation, goal: string, hist
 					id: `type:${element.id}`,
 					kind: "target",
 					operation,
-					label: `Enter text in ${JSON.stringify(element.name)}; current value=${JSON.stringify(element.value)}`,
+					label: `Enter text in ${JSON.stringify(element.name)}${stateDescription(element)}`,
 					target,
 					value: element.value,
+					hasValue: element.hasValue,
 					textPurpose: "field",
 				});
 				continue;
@@ -70,6 +80,20 @@ export function buildCandidateSpace(observation: Observation, goal: string, hist
 		}
 	}
 
+	if (options.credentials) {
+		for (const form of observation.credentialForms) {
+			const fields = form.fields.filter((field) => !field.hasValue);
+			if (!fields.length) continue;
+			candidates.push({
+				id: `credentials:${form.id}`,
+				kind: "credential",
+				operation: "USE_CREDENTIALS",
+				label: `Use a vault credential for ${JSON.stringify(form.name)} fields: ${fields.map((field) => field.name).join(", ")}`,
+				credentialForm: { ...form, fields },
+			});
+		}
+	}
+
 	const scrollAmount = Math.max(1, Math.ceil(observation.scroll.viewport / 120));
 	if (!navigationOnly && observation.scroll.y + observation.scroll.viewport < observation.scroll.height - 2) {
 		candidates.push({
@@ -116,7 +140,9 @@ export function buildCandidateSpace(observation: Observation, goal: string, hist
 		if (hasForwardHistory(history)) candidates.push({ id: "history:forward", kind: "history", operation: "FORWARD", label: "Go forward one page" });
 		candidates.push({ id: "history:reload", kind: "history", operation: "RELOAD", label: "Reload the current page" });
 	}
-	candidates.push({ id: "done", kind: "terminal", operation: "DONE", label: "Every requirement is visibly satisfied" });
+	if (!(isAuthenticationGoal(goal) && observation.credentialForms.length > 0)) {
+		candidates.push({ id: "done", kind: "terminal", operation: "DONE", label: "Every requirement is visibly satisfied" });
+	}
 	candidates.push({ id: "blocked", kind: "terminal", operation: "BLOCKED", label: "No supported operation can make progress safely" });
 
 	const byOperation = new Map<Operation, JevCandidate[]>();
@@ -133,6 +159,10 @@ export function buildCandidateSpace(observation: Observation, goal: string, hist
 	return { candidates, byId: new Map(candidates.map((candidate) => [candidate.id, candidate])), byOperation, elements };
 }
 
+export function isAuthenticationGoal(goal: string): boolean {
+	return /\b(?:sign[ -]?in(?:to)?|log[ -]?in(?:to)?|login|authenticate)\b/i.test(goal);
+}
+
 export function extractLiteralUrls(goal: string): string[] {
 	const urls = new Set<string>();
 	for (const match of goal.matchAll(/https?:\/\/[^\s<>"']+/gi)) {
@@ -152,7 +182,7 @@ function isElementOperation(operation: Operation): operation is ElementOperation
 }
 
 function isExcludedControl(element: Observation["elements"][number]): boolean {
-	return FILE_CONTROL.test(element.name) || (element.operations.includes("TYPE_TEXT") && SECRET_FIELD.test(element.name));
+	return element.sensitive === true || FILE_CONTROL.test(element.name) || (element.operations.includes("TYPE_TEXT") && SECRET_FIELD.test(element.name));
 }
 
 function hasForwardHistory(history: readonly HistoryEntry[]): boolean {
@@ -172,7 +202,7 @@ function hasForwardHistory(history: readonly HistoryEntry[]): boolean {
 
 function stateDescription(element: Observation["elements"][number]): string {
 	const states = [
-		element.value ? `value=${JSON.stringify(element.value)}` : undefined,
+		element.hasValue === undefined ? (element.value ? `value=${JSON.stringify(element.value)}` : undefined) : `has_value=${element.hasValue}`,
 		element.checked === undefined ? undefined : `checked=${element.checked}`,
 		element.selected === undefined ? undefined : `selected=${element.selected}`,
 		element.expanded === undefined ? undefined : `expanded=${element.expanded}`,

packages/browser-loop/examples/jev-system-one/actions.unit.ts

@@ -3,7 +3,7 @@ import { describe, it } from "node:test";
 import type { BrowserAction } from "../../src/core/actions/browser";
 import type { BrowserExecutor } from "../../src/core/translator/browser";
 import { ObservationChangedError } from "../../src/core/translator/browser-observation";
-import { buildCandidateSpace, extractLiteralUrls } from "./actions";
+import { buildCandidateSpace, extractLiteralUrls, isAuthenticationGoal } from "./actions";
 import { ExecutorBrowserRuntime, observationFromElements } from "./browser";
 import type { ElementOperation, HistoryEntry, ObservationElement } from "./types";
 
@@ -113,6 +113,13 @@ describe("Jev candidate space", () => {
 		assert.equal(buildCandidateSpace(observation, "Open Google Flights").byOperation.get("NAVIGATE")?.[0]?.textPurpose, "navigation");
 	});
 
+	it("recognizes common authentication goal wording", () => {
+		assert.equal(isAuthenticationGoal("Sign in to GitHub"), true);
+		assert.equal(isAuthenticationGoal("Sign into GitHub"), true);
+		assert.equal(isAuthenticationGoal("Log into my account"), true);
+		assert.equal(isAuthenticationGoal("Open the homepage"), false);
+	});
+
 	it("treats internal startup pages as navigation-only", () => {
 		const newTab = observationFromElements({
 			url: "chrome://newtab/",

packages/browser-loop/examples/jev-system-one/agent.ts

@@ -1,8 +1,10 @@
 import type { BrowserAction } from "../../src/core/actions/browser";
 import { buildCandidateSpace } from "./actions";
+import { CredentialBlockedError } from "./types";
 import type {
 	AgentResult,
 	BrowserRuntime,
+	CredentialBroker,
 	HistoryEntry,
 	JevCandidate,
 	JevCandidateSpace,
@@ -22,6 +24,7 @@ export async function runAgent(options: {
 	browser: BrowserRuntime;
 	policy: JevPolicy;
 	textResolver?: TextResolver;
+	credentialBroker?: CredentialBroker;
 	maxSteps?: number;
 	onDecision?: (trace: AgentResult["steps"][number]) => void;
 	onFreshness?: (trace: { step: number; latencyMs: number; changed: boolean }) => void;
@@ -39,7 +42,7 @@ export async function runAgent(options: {
 
 	for (let step = 0; step < (options.maxSteps ?? MAX_STEPS); step++) {
 		const rejected = rejectedByState.get(observation.interactionFingerprint);
-		const space = rejectCandidates(buildCandidateSpace(observation, options.goal, history), rejected);
+		const space = rejectCandidates(buildCandidateSpace(observation, options.goal, history, { credentials: options.credentialBroker !== undefined }), rejected);
 		const decision = await options.policy.decide({ goal: options.goal, observation, space, history });
 		usage.calls += 1;
 		usage.inputTokens += decision.inputTokens;
@@ -95,18 +98,28 @@ export async function runAgent(options: {
 		let executeMs = 0;
 		try {
 			const resolveStarted = performance.now();
-			lowered = await lowerCandidate(candidate, options.goal, observation, history, options.textResolver);
+			lowered = candidate.kind === "credential"
+				? undefined
+				: await lowerCandidate(candidate, options.goal, observation, history, options.textResolver);
 			resolveMs = performance.now() - resolveStarted;
-			if (!lowered) {
+			if (!lowered && candidate.kind !== "credential") {
 				status = "blocked";
 				reason = `No text value was available for ${candidate.label}`;
 				break;
 			}
 			const executeStarted = performance.now();
-			if (lowered.kind === "target") await options.browser.executeTarget(lowered.candidate, lowered.value);
-			else await options.browser.execute(lowered.action);
+			if (candidate.kind === "credential") {
+				if (!options.credentialBroker) throw new Error("A credential broker is required for credential actions");
+				await options.credentialBroker.use({ goal: options.goal, candidate, observation, history, browser: options.browser });
+			} else if (lowered?.kind === "target") await options.browser.executeTarget(lowered.candidate, lowered.value);
+			else if (lowered) await options.browser.execute(lowered.action);
 			executeMs = performance.now() - executeStarted;
 		} catch (error) {
+			if (error instanceof CredentialBlockedError) {
+				status = "blocked";
+				reason = error.message;
+				break;
+			}
 			if (/stale.*ref|ref.*stale|page changed|target changed/i.test(errorMessage(error))) {
 				observation = await options.browser.observe();
 				continue;
@@ -125,7 +138,7 @@ export async function runAgent(options: {
 			operation: candidate.operation,
 			candidateId: candidate.id,
 			label: candidate.label,
-			...(lowered.value === undefined ? {} : { value: lowered.value }),
+			...(lowered?.value === undefined ? {} : { value: lowered.value }),
 			pageChanged: successor.fingerprint !== observation.fingerprint,
 			url: successor.url,
 		};