Skip to content

Make the Bun delivery fallback test pass consistently #1232

Open

Description

@taras

Story

The delivery workflow’s Fable-to-Astra fallback test should give the same result each time it runs. It must verify that the workflow presents one hold when neither reviewer can complete.

Evidence

Bun shard 6/10 ran bun run test:bun 6/10 with Bun 1.4.0 on Linux at 92c27cd3c9ff69092a17ed536d24f39918ae9e7e. The Delivery Markdown test A-H required review follows Fable, one independent Astra replacement, then a retained Elicit hold failed:

Expected values to be strictly deep-equal:
2 !== 1
(fail) adversarial implementation example > runs the Delivery Markdown journeys without native models

The full first log is linked above (SHA-256 661e9bb04b920384279e6f8fa0a82f862fd9e736e97f806d3e9f5500f4801052). It does not identify which count assertion failed.

The same revision passed the focused journey unchanged on macOS arm64 with Bun 1.3.14: one test passed, zero failed, 168.47 seconds total.

bun test --timeout=300000 packages/cli/tests/adversarial-delivery.test.ts

The preceding Linux Bun shard passed with the same fixture source. The intervening change was in ACP load diagnostics. This comparison does not establish the cause of the count mismatch.

The failed-jobs-only CI retry passed the same Linux Bun shard on the identical commit. No code changed between these two CI attempts.

Acceptance

Identify the specific count and conditions that let it differ. Keep the one-review and one-hold assertions, and show that the journey passes repeatedly under the affected Linux Bun CI conditions.

Activity

  1. added
    flakeIntermittent or timing-sensitive failure that can pass without a code change
    on Nov 10, 2026
  2. taras commented on Nov 11, 2026

    @taras
    OwnerAuthor

    The same Delivery fallback assertion recurred in PR #1240's first Linux Bun 1.4.0 shard 6/10, command bun run test:bun 6/10, at head 7450ca217d2fd03d65e59a041809cbf21954d419.

    A-H required review follows Fable, one independent Astra replacement, then a retained Elicit hold failed with 2 !== 1. The outer Delivery Markdown journey exited 1 after 49109.91 ms. All 46 assigned files ran; the shard reports adversarial-delivery.test.ts as its first failing file. Later expected-refusal diagnostics in other passing files are not additional failed tests.

    The complete first output was preserved before any retry at /private/tmp/xmd-transcript-1227-20261110/ci-bun6-first-api-130254873235.log, 495017 bytes, SHA-256 75cc4a271b519c68b8d54a8c54cfdbe6455308b3bd1ab6e808084ec7fc39379f.

    #1240 changes only repl-wheel-scroll.test.ts. This recurrence supplies evidence for the existing Delivery issue; it establishes no new cause for the count mismatch. Its Transcript shard passed separately.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    flakeIntermittent or timing-sensitive failure that can pass without a code change

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions