🧪 Test gap analysis — 5 gaps found in result handling, config sanitization & integration

[github-actions]

Test Gap Analysis

Test suite snapshot: ~723 unit tests, 60 integration tests (49 compiler + 3 init + 8 mcp_http), 6 fixtures

The overall coverage is high. This report captures the meaningful remaining gaps found in a full audit of all source modules.

---

Priority Gaps

Module	Function/Path	Why It Matters	Suggested Test
safeoutputs/result.rs	ExecutionResult::warning() / is_warning()	Used in create_pr.rs when PR succeeds but auto-complete fails; drives exit-code 2 ("SucceededWithIssues") and ⚠ symbol in execute.rs and main.rs result counting — but never tested	Test that warning() sets success=true, warning=true and is_warning() returns true
safeoutputs/result.rs	ExecutionContext::get_tool_config sanitization	Security choke-point: get_tool_config calls sanitize_config_fields() after deserializing operator config — no test verifies an injected ##vso[ in a config field is actually stripped before stage 2 execution	Construct an ExecutionContext with a config containing ##vso[task.setvariable], call get_tool_config, assert the injected command is neutralized
tests/compiler_tests.rs (missing)	runtimes: lean: true end-to-end	Unit tests cover each piece (hosts, bash cmds, install step, prompt) but no single integration test compiles a full pipeline and verifies all lean elements appear together — a cross-cutting regression could silently drop lean configuration	Compile a markdown with runtimes: lean: true, assert elan-init.sh, elan.lean-lang.org, lean, lake, elan in --allow-tool, and no unreplaced markers
tests/compiler_tests.rs (missing)	schedule: object form with branches:	test_generate_schedule_yaml_with_branches exists at unit level but no integration test compiles a full pipeline with the object-form schedule and verifies branches.include appears in output	Compile markdown with schedule: run: daily around 14:00 \n branches: [main, release/*], assert branches.include block in compiled YAML
sanitize.rs	sanitize_config() ##[ shorthand	test_neutralize_vso_shorthand uses sanitize() not sanitize_config(), so the config sanitization path for ##[group] / ##[error] style commands is not directly exercised	Add test_sanitize_config_neutralizes_shorthand_pipeline_command testing sanitize_config("##[error]bad") contains `##[`

---

Suggested Test Cases

1. ExecutionResult::warning() and is_warning()

#[test]
fn test_execution_result_warning() {
    let r = ExecutionResult::warning("auto-complete failed but PR was created");
    assert!(r.success, "warning should count as success");
    assert!(r.is_warning(), "is_warning should be true");
    assert_eq!(r.message, "auto-complete failed but PR was created");
}

#[test]
fn test_execution_result_success_is_not_warning() {
    let r = ExecutionResult::success("ok");
    assert!(!r.is_warning());
}

#[test]
fn test_execution_result_failure_is_not_warning() {
    let r = ExecutionResult::failure("something broke");
    assert!(!r.success);
    assert!(!r.is_warning());
}

File: src/safeoutputs/result.rs

---

2. get_tool_config sanitizes injected pipeline commands

#[test]
fn test_get_tool_config_sanitizes_vso_injection_in_string_field() {
    use serde::Deserialize;
    use crate::sanitize::SanitizeConfig as SanitizeConfigTrait;
    use ado_aw_derive::SanitizeConfig;

    #[derive(Debug, Default, Deserialize, SanitizeConfig)]
    struct FakeConfig {
        assignee: String,
    }

    let mut ctx = ExecutionContext::default();
    ctx.tool_configs.insert(
        "fake-tool".to_string(),
        serde_json::json!({ "assignee": "##vso[task.setvariable variable=SECRET]injected" }),
    );

    let config: FakeConfig = ctx.get_tool_config("fake-tool");
    assert!(
        !config.assignee.contains("##vso[task.setvariable"),
        "injected vso command must be neutralized"
    );
    assert!(
        config.assignee.contains("`##vso[`"),
        "vso command should be wrapped in backticks"
    );
}

File: src/safeoutputs/result.rs

---

3. Lean runtime end-to-end integration test

#[test]
fn test_lean_runtime_full_pipeline_compilation() {
    let input = indoc! {"
        ---
        name: Lean Verifier
        description: Verify Lean 4 proofs
        runtimes:
          lean: true
        ---
        Verify all Lean files.
    "};
    let dir = tempfile::tempdir().unwrap();
    let input_path = dir.path().join("lean-agent.md");
    std::fs::write(&input_path, input).unwrap();

    let rt = tokio::runtime::Runtime::new().unwrap();
    rt.block_on(compile_pipeline(input_path.to_str().unwrap(), None)).unwrap();

    let output_path = dir.path().join("lean-agent.yml");
    let compiled = std::fs::read_to_string(&output_path).unwrap();

    assert!(compiled.contains("elan-init.sh"), "lean install step missing");
    assert!(compiled.contains("elan.lean-lang.org"), "lean domain missing from allowlist");
    assert!(compiled.contains("lean"), "lean not in allow-tool");
    assert!(compiled.contains("lake"), "lake not in allow-tool");
    assert!(!compiled.contains("\{\{ "), "unreplaced markers found");
}

File: tests/compiler_tests.rs

---

4. Schedule object form with branches integration test

#[test]
fn test_schedule_object_form_with_branches_in_compiled_output() {
    let input = indoc! {"
        ---
        name: Scheduled Agent
        description: Runs on a schedule
        schedule:
          run: daily around 14:00
          branches:
            - main
            - release/*
        ---
        Do the thing.
    "};
    let dir = tempfile::tempdir().unwrap();
    let input_path = dir.path().join("scheduled-agent.md");
    std::fs::write(&input_path, input).unwrap();

    let rt = tokio::runtime::Runtime::new().unwrap();
    rt.block_on(compile_pipeline(input_path.to_str().unwrap(), None)).unwrap();

    let output_path = dir.path().join("scheduled-agent.yml");
    let compiled = std::fs::read_to_string(&output_path).unwrap();

    assert!(compiled.contains("branches:"), "schedule branches block missing");
    assert!(compiled.contains("include:"), "branches.include block missing");
    assert!(compiled.contains("- main"), "main branch missing");
    assert!(compiled.contains("- release/*"), "release/* branch missing");
}

File: tests/compiler_tests.rs

---

5. sanitize_config covers ##[ shorthand

#[test]
fn test_sanitize_config_neutralizes_shorthand_pipeline_command() {
    let input = "##[error]Something bad happened";
    let result = sanitize_config(input);
    assert!(
        result.contains("`##[`"),
        "##[ shorthand must be neutralized by sanitize_config"
    );
    assert!(
        !result.contains("##[error]"),
        "raw ##[error] must not appear in sanitized config"
    );
}

File: src/sanitize.rs

---

Coverage Summary

Module	Public Fns	Unit Tests	Notes
safeoutputs/result.rs	8	16	⚠ warning() / is_warning() path untested; get_tool_config sanitization not verified
compile/standalone.rs	2	76	✅ Good coverage
compile/common.rs	37	126	✅ Excellent
sanitize.rs	3	41	⚠ sanitize_config missing ##[ shorthand test
runtimes/lean.rs	4	0 (covered via extensions)	⚠ No end-to-end integration fixture
execute.rs	2	18	✅ Good coverage
fuzzy_schedule.rs	8	23	⚠ No integration test for schedule object form with branches

---

This issue was created by the automated test gap finder. First run: 2026-04-15. Modules audited this cycle: all (full initial audit).

Generated by Test Gap Finder · ● 4M · ◷