Skip to content

Add support for the --dry-run flag during agent apply#160

Open
geoffjay wants to merge 1 commit intomainfrom
issue-133-dry-run
Open

Add support for the --dry-run flag during agent apply#160
geoffjay wants to merge 1 commit intomainfrom
issue-133-dry-run

Conversation

@geoffjay
Copy link
Owner

@geoffjay geoffjay commented Mar 5, 2026

The --dry-run flag already existed but now provides detailed output showing:

  • Resolved agent configurations (working_dir, shell, model, env vars, etc.)
  • Resolved workflow configurations (agent reference, source details, poll interval)
  • Dependency order (agents first, then workflows)
  • Summary counts of resources to be created

JSON output includes full resolved configurations for programmatic use.
The dry-run mode validates all templates without making HTTP calls to
the orchestrator, allowing users to preview changes before applying.

Closes #133

Co-Authored-By: Claude Opus 4.6 noreply@anthropic.com

The --dry-run flag already existed but now provides detailed output showing:
- Resolved agent configurations (working_dir, shell, model, env vars, etc.)
- Resolved workflow configurations (agent reference, source details, poll interval)
- Dependency order (agents first, then workflows)
- Summary counts of resources to be created

JSON output includes full resolved configurations for programmatic use.
The dry-run mode validates all templates without making HTTP calls to
the orchestrator, allowing users to preview changes before applying.

Closes #133

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@codecov
Copy link

codecov bot commented Mar 5, 2026

Codecov Report

❌ Patch coverage is 0% with 132 lines in your changes missing coverage. Please review.
✅ Project coverage is 22.42%. Comparing base (fdd16e6) to head (17bc4e1).
⚠️ Report is 94 commits behind head on main.

Files with missing lines Patch % Lines
crates/cli/src/commands/apply.rs 0.00% 132 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #160      +/-   ##
==========================================
- Coverage   23.03%   22.42%   -0.61%     
==========================================
  Files          50       50              
  Lines        4620     4745     +125     
==========================================
  Hits         1064     1064              
- Misses       3556     3681     +125     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@geoffjay geoffjay added the review-agent Used to invoke a review by an agent tracking this label label Mar 9, 2026
Copy link
Owner Author

@geoffjay geoffjay left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review: Changes Needed 🔴

The output improvements are a clear win — showing resolved configs instead of just names is much more useful. However there are two blocking issues and a few non-blocking ones.


🔴 Blocking #1: non-exhaustive match on SourceTemplate (compile error)

SourceTemplate has two variants: GithubIssues and GithubPullRequests. The new dry-run match arms only cover GithubIssues, so the compiler will reject this with:

error[E0004]: non-exhaustive patterns: `GithubPullRequests { .. }` not covered

Affected sites:

  1. apply_workflow_file — new JSON branch (match &tmpl.source { GithubIssues … })
  2. apply_directory — workflow JSON builder (match &t.source { GithubIssues … })
  3. apply_directory — workflow human-readable output (match &tmpl.source { GithubIssues … })

The live code path (~line 310) already handles both variants correctly and can be used as a reference:

SourceTemplate::GithubPullRequests { owner, repo, labels, state } => {
    serde_json::json!({
        "type": "github_pull_requests",
        "owner": owner, "repo": repo, "labels": labels, "state": state,
    })
}

Each match needs a GithubPullRequests arm at all three sites.


🔴 Blocking #2: apply_workflow_file dry-run still makes HTTP calls

The PR description states:

"The dry-run mode validates all templates without making HTTP calls to the orchestrator"

This is not true for apply_workflow_file. The function structure is:

// line 290 — HTTP call happens here, before the dry-run check
let agent = find_agent_by_name(client, &tmpl.agent).await?...;

// ...status check...

// line 303 — dry-run check happens after the HTTP call
if dry_run {
    ...
    "agent_id": agent.id.to_string(),  // relies on the HTTP response
    ...
}

find_agent_by_name is called unconditionally before if dry_run, so running agent apply --dry-run some-workflow.yml with no orchestrator running will fail with a connection error. The new JSON dry-run output also exposes agent_id, making it explicitly depend on the HTTP response.

The fix is to move the dry-run early-return to before the find_agent_by_name call — for the JSON case, omit agent_id (the name is sufficient since the UUID isn't known without an HTTP call), and for the text case, show the agent name as a symbolic reference.


🟡 Non-blocking suggestions

1. apply_directory workflow JSON uses t.prompt_template directly

"prompt_template": t.prompt_template,

WorkflowTemplate.prompt_template is Option<String>. If a workflow uses prompt_template_file instead of an inline prompt_template, this field will be null in the JSON output even though resolve_prompt would return the full resolved string. Phase 0 already calls resolve_prompt (line 385); caching its result in the workflow_templates vec would let the dry-run JSON show the actual resolved value.

2. working_dir is not resolved in JSON output

In both apply_agent_file and apply_directory the JSON shows:

"working_dir": &tmpl.working_dir,  // may be "."

The live apply_agent function resolves . to current_dir() before sending to the orchestrator. The dry-run JSON should show the same resolved path for an accurate preview.

3. Human-readable dry-run for apply_workflow_file is not enhanced

The JSON branch now shows source details and poll interval, but the else (text) branch still only prints the old valid (agent → UUID) line. Given that the PR enhances text output for agents and directory-level workflows, a similar treatment for single workflow files would be consistent.

@geoffjay geoffjay removed the review-agent Used to invoke a review by an agent tracking this label label Mar 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add --dry-run flag to CLI apply command

1 participant