Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/guides/commands.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ The other commands are direct only.
| `npm run call` | `npm run call -- <org> -a <assistant>` or `-s <squad>` | Talk to an assistant or squad from your terminal. |
| `npm run sim` | `npm run sim -- <org> --suite <name> --target <name>` | Run a simulation suite against deployed resources. Exits 0 passed, 1 failed, 3 incomplete. |
| `npm run check` | `npm run check -- <check>` or `--all`, `[--dry-run]` | Run [PR checks](pr-checks.md) against local files, nothing deployed. Exits 0 passed, 1 failed, 2 config or build error, 3 incomplete. |
| `npm run promote` | `npm run promote -- --pipeline <p> --from <org> --to <org> [--apply]` | Plan, or apply, a [promotion](promotion.md) between orgs. |
| `npm run promote` | `npm run promote -- --pipeline <p> --from <org> --to <org> [--apply] [--no-rollout \| --rollout-override]` | Plan, or apply, a [promotion](promotion.md) between orgs. Into an org with a [gradual rollout](promotion.md#gradual-rollout-optional), `--no-rollout` publishes at once and `--rollout-override` pins over a split set outside gitops. |
| `npm run build` | `npm run build` | Type-check the code and tests. |
| `npm test` | `npm test` | Run the test suite. |

Expand Down
34 changes: 34 additions & 0 deletions docs/guides/promotion.md
Original file line number Diff line number Diff line change
Expand Up @@ -170,6 +170,36 @@ The org needs traffic splitting, which Vapi enables per org during its beta.
An `--apply` run checks every rollout org first and stops, with nothing
promoted, if one doesn't have it.

### What a promotion into a rollout org does

For each changed assistant that already exists in the org:

1. **Pin.** Before publishing, it writes a traffic allocation holding the
version that serves calls now at 100%. Without it, the publish would move
every call at once.
2. **Publish.** The apply publishes the new version, which lands at 0%.
3. **Step 1.** It moves the first step's share (1% by default) to the new
version. If nothing was published (the files matched), it undoes the pin.

The rollout workflow then moves the ramp one step at a time (see below).
Every write carries a `gitops-rollout …` note in the allocation's
description, which is the ramp's only state; the traffic tab shows it.

Not everything is split. A new assistant goes live when it's created; squads,
tools and structured outputs aren't versioned per call, so a change to them
reaches 100% of calls on apply. The plan output lists each of these.

The promotion refuses, before the gate and with nothing written, when an
assistant has a split someone set by hand (a manual split wins) or is managed
by Vapi for CX. To pin over a manual split on purpose, promote that one
transition with `--rollout-override` (the `override` dispatch input): it pins
the version with the most traffic, never the canary.

`--no-rollout` (the `no-rollout` dispatch input) publishes at once. Use it
for a revert: it also releases an assistant gitops was holding, so the
reverted version takes every call. An assistant whose files didn't change,
or a split set by hand, is left as it is.

## Rolling Back a Promotion

Treat a promotion rollback as a new, auditable Git change: revert the source
Expand All @@ -180,6 +210,10 @@ git revert <promotion-commit>
npm run promote -- --pipeline release --from dev --to staging --apply
```

Into an org with a gradual rollout, add `--no-rollout` so the revert goes
live at once instead of ramping. To stop a ramp in progress without a new
promotion, abort it (see the rollout workflow below).

This is distinct from `npm run rollback`, which restores a single org from a
local pre-deploy snapshot. Because promotion uses the pipeline's scoped mirror
boundary, reverting a resource creation also removes that promoted resource
Expand Down
195 changes: 183 additions & 12 deletions src/promote-cmd.ts
Original file line number Diff line number Diff line change
Expand Up @@ -14,12 +14,28 @@ import {
} from "./promotion-gate.ts";
import type { RolloutApi } from "./rollout-api.ts";
import { rolloutApiCreate } from "./rollout-api.ts";
import type {
RolloutAssistantRef,
RolloutPin,
RolloutRead,
} from "./rollout-promote.ts";
import {
rolloutFinish,
rolloutPinAll,
rolloutPreflight,
rolloutRelease,
rolloutReleaseBaseline,
rolloutSnapshot,
rolloutUnpinnedReport,
} from "./rollout-promote.ts";
import type { PromotionChange } from "./promotion.ts";
import {
promotionConfigParse,
promotionPlanApply,
promotionPlanBuild,
promotionStateParse,
promotionTransitionValidate,
resourceType,
} from "./promotion.ts";

interface PromotionArguments {
Expand All @@ -28,6 +44,10 @@ interface PromotionArguments {
target?: string;
all: boolean;
apply: boolean;
// Skip the gradual rollout: publish at once, as a revert should.
noRollout: boolean;
// Pin over a split set outside gitops instead of refusing.
rolloutOverride: boolean;
}

interface PromotionTransition {
Expand Down Expand Up @@ -67,7 +87,12 @@ interface PromotionGates {
}

function argumentsParse(args: string[]): PromotionArguments {
const parsed: PromotionArguments = { all: false, apply: false };
const parsed: PromotionArguments = {
all: false,
apply: false,
noRollout: false,
rolloutOverride: false,
};
for (let index = 0; index < args.length; index++) {
const argument = args[index];
if (argument === "--all") {
Expand All @@ -78,6 +103,14 @@ function argumentsParse(args: string[]): PromotionArguments {
parsed.apply = true;
continue;
}
if (argument === "--no-rollout") {
parsed.noRollout = true;
continue;
}
if (argument === "--rollout-override") {
parsed.rolloutOverride = true;
continue;
}
if (argument === "--pipeline" && args[index + 1]) {
parsed.pipeline = args[++index];
continue;
Expand All @@ -98,7 +131,15 @@ function argumentsParse(args: string[]): PromotionArguments {
);
if (!parsed.all && (!parsed.pipeline || !parsed.source || !parsed.target))
throw new Error(
"Usage: npm run promote -- --pipeline <name> --from <org> --to <org> [--apply]",
"Usage: npm run promote -- --pipeline <name> --from <org> --to <org> [--apply] [--no-rollout | --rollout-override]",
);
if (parsed.noRollout && parsed.rolloutOverride)
throw new Error("--no-rollout cannot be combined with --rollout-override");
// Overriding a manual split is a decision about one transition, never a
// blanket one across every org.
if (parsed.rolloutOverride && parsed.all)
throw new Error(
"--rollout-override needs --pipeline, --from and --to; it cannot be combined with --all",
);
return parsed;
}
Expand Down Expand Up @@ -254,6 +295,49 @@ function stateLoad(org: string) {
return promotionStateParse(readFileSync(path, "utf8"));
}

// The changed assistants a rollout can ramp: creates and updates with a UUID
// in the target's state. Deletes have nothing to ramp, and an assistant with
// no UUID is created by the apply, so it goes live on creation.
function rolloutRefsGet(
target: string,
changes: PromotionChange[],
): RolloutAssistantRef[] {
const assistants = stateLoad(target).assistants;
const refs: RolloutAssistantRef[] = [];
for (const change of changes) {
if (change.kind === "delete") continue;
const resource = resourceType(change.path);
if (resource?.type !== "assistants") continue;
const uuid = assistants[resource.id]?.uuid;
if (!uuid) {
console.log(` rollout ${resource.id}: new in ${target}, live on create`);
continue;
}
refs.push({ id: resource.id, uuid });
}
return refs;
}

// What a promotion publishes at 100% even under a rollout: only direct
// assistant calls are split, and tools and structured outputs aren't
// versioned.
function rolloutUnsplitPrint(changes: PromotionChange[]): void {
for (const change of changes) {
const type = resourceType(change.path)?.type;
if (type && type !== "assistants")
console.log(` rollout ${change.path}: not split, live at 100% on apply`);
}
}

// The target's other assistants, whose versions the unpinned-publish report
// compares across the apply.
function othersGet(target: string, pins: RolloutPin[]): RolloutAssistantRef[] {
const pinned = new Set(pins.map((pin) => pin.read.ref.uuid));
return Object.entries(stateLoad(target).assistants)
.filter(([, entry]) => entry?.uuid && !pinned.has(entry.uuid))
.map(([id, entry]) => ({ id, uuid: entry.uuid }));
}

async function transitionRun(
config: PromotionConfig,
transition: PromotionTransition,
Expand All @@ -262,6 +346,7 @@ async function transitionRun(
allowEmptySourceDeletion: boolean,
deps: PromotionDeps,
gates: PromotionGates,
args: PromotionArguments,
): Promise<boolean> {
const run = deps.childRun;
if (apply) {
Expand Down Expand Up @@ -299,6 +384,24 @@ async function transitionRun(
if (check && !apply)
console.log(await promotionGatePlanLine(ROOT_DIR, check));
if (!apply) return false;
const rollout = config.orgs[transition.target]?.rollout;
const api = rollout
? deps.rolloutApi(orgConnection(config, transition.target, tokens))
: undefined;
const refs = rollout ? rolloutRefsGet(transition.target, plan.changes) : [];
const ramp = api && rollout && !args.noRollout ? { api, rollout } : undefined;
if (ramp) rolloutUnsplitPrint(plan.changes);
// A fast refusal before the gate's simulations, which cost minutes.
let reads: RolloutRead[] = ramp
? await rolloutPreflight(
ramp.api,
transition.target,
refs,
args.rolloutOverride,
)
: [];
// Decided before the gate block, which marks the source passed.
const gateRuns = Boolean(check && !gates.passed.has(transition.source));
if (check && !gates.passed.has(transition.source)) {
console.log(` check running ${check.name} in ${transition.source}…`);
const result = await deps.checkRun(
Expand All @@ -311,19 +414,85 @@ async function transitionRun(
);
gates.passed.add(transition.source);
}
// The gate can run for hours while the ticker or an abort writes, so read
// again: pins and undos must start from what is current now.
if (ramp && gateRuns)
reads = await rolloutPreflight(
ramp.api,
transition.target,
refs,
args.rolloutOverride,
);
await promotionPlanApply(plan);
const changedPaths = plan.changes.map(
(change) => `resources/${transition.target}/${change.path}`,
);
run(
"src/apply.ts",
transition.target,
orgConnection(config, transition.target, tokens),
["--force", "--allow-new-files", "--resolve=ours", ...changedPaths],
);
appliedPathsRecord(transition.target);
// The target's files just changed, so an earlier pass no longer covers it.
gates.passed.delete(transition.target);
const releaseBaseline =
api && rollout && args.noRollout
? await rolloutReleaseBaseline(api, refs)
: undefined;
const pins: RolloutPin[] = ramp
? await rolloutPinAll(
ramp.api,
transition.target,
reads,
args.rolloutOverride,
)
: [];
const others = ramp ? othersGet(transition.target, pins) : [];
let childError: unknown;
const failures: string[] = [];
try {
const before = ramp ? await rolloutSnapshot(ramp.api, others) : undefined;
try {
run(
"src/apply.ts",
transition.target,
orgConnection(config, transition.target, tokens),
["--force", "--allow-new-files", "--resolve=ours", ...changedPaths],
);
} catch (error) {
childError = error;
}
if (ramp && before) await rolloutUnpinnedReport(ramp.api, others, before);
if (api && releaseBaseline)
failures.push(...(await rolloutRelease(api, refs, releaseBaseline)));
} finally {
if (ramp)
failures.push(
...(await rolloutFinish({
api: ramp.api,
pins,
steps: ramp.rollout.steps,
childFailed: childError !== undefined,
})),
);
}
if (childError === undefined) {
// Recorded before any rollout failure is thrown: the org has the changes,
// so the committed state must have its files. With no rollout failure to
// report alongside it, a recording failure throws as it is.
if (failures.length === 0) appliedPathsRecord(transition.target);
else
try {
appliedPathsRecord(transition.target);
} catch (error) {
failures.unshift(
error instanceof Error ? error.message : String(error),
);
}
// The target's files just changed, so an earlier pass no longer covers it.
gates.passed.delete(transition.target);
}
if (childError !== undefined && failures.length === 0) throw childError;
if (failures.length > 0)
throw new Error(
[
...(childError instanceof Error ? [childError.message] : []),
`Gradual rollout into ${transition.target}:`,
...failures.map((failure) => ` - ${failure}`),
].join("\n"),
);
return plan.changes.some((change) => change.kind === "delete");
}

Expand Down Expand Up @@ -360,7 +529,8 @@ export async function promotionCommandRun(
// reviewed authorization forward so the same deletion can reach later orgs.
const deletionAuthorizedSources = new Set<string>();
const transitions = transitionsBuild(config, parsed);
if (parsed.apply) await rolloutOrgsProbe(config, transitions, tokens, deps);
if (parsed.apply && !parsed.noRollout)
await rolloutOrgsProbe(config, transitions, tokens, deps);
for (const transition of transitions) {
const sourceKey = `${transition.pipeline}:${transition.source}`;
const deleted = await transitionRun(
Expand All @@ -371,6 +541,7 @@ export async function promotionCommandRun(
deletionAuthorizedSources.has(sourceKey),
deps,
gates,
parsed,
);
if (deleted)
deletionAuthorizedSources.add(
Expand Down
2 changes: 1 addition & 1 deletion src/promotion.ts
Original file line number Diff line number Diff line change
Expand Up @@ -354,7 +354,7 @@ function ignored(path: string, patterns: string[]): boolean {
return patterns.some((pattern) => glob(pattern, stem));
}

function resourceType(path: string): { type: ResourceType; id: string } | null {
export function resourceType(path: string): { type: ResourceType; id: string } | null {
for (const [type, folder] of Object.entries(FOLDER_MAP)) {
if (!path.startsWith(`${folder}/`)) continue;
const id = path.slice(folder.length + 1).replace(/\.(yml|yaml|md|ts)$/, "");
Expand Down
Loading
Loading