Skip to content

test(e2e): require cross-observer agreement on sentinel fault slot#23513

Merged
PhilWindle merged 1 commit into
merge-train/spartanfrom
cb/4bca97200b39
May 23, 2026
Merged

test(e2e): require cross-observer agreement on sentinel fault slot#23513
PhilWindle merged 1 commit into
merge-train/spartanfrom
cb/4bca97200b39

Conversation

@AztecBot
Copy link
Copy Markdown
Collaborator

Why

PR #23344 (merge-train/spartan) was dequeued because e2e_p2p/sentinel_status_slash.parallel.test.ts "slashes the proposer with INACTIVITY when checkpoint validation records invalid" flaked in shard ci/x3-full (run, log).

From the failing log:

  • 14:36:28 malicious validator-1 broadcasts INVALID checkpoint for slot 11
  • 14:37:24 only validator-2 records checkpoint-invalid at slot 11
  • 14:38:29–31 when the malicious node is selected again at slot 27, validators 2/3/4/5/6 all record checkpoint-invalid at slot 27
  • Test then failed with TimeoutError: Timeout awaiting sentinel status for 0x90f7…906 at slot 11

findObservedStatusSlot was polling only honestObservers[0] (validator-2), which had picked up the malicious proposer's gossip earlier than the other observers. It returned slot 11. assertAllObserversSentinelStatus then timed out polling the rest of the observers for a slot they had never recorded — they only recorded checkpoint-invalid at the next occurrence (slot 27).

What

Change findObservedStatusSlot to require cross-observer agreement: it now takes the full observerNodes list and returns the earliest slot where every honest observer has recorded expectedStatus. That guarantees the downstream assertAllObserversSentinelStatus call has a slot every observer can satisfy.

Times out — and fails — if no common fault slot is ever recorded, so a genuine detection failure is still caught.

Both flake-prone tests (checkpoint-unvalidated and checkpoint-invalid) use the same helper and benefit equally.


Created by claudebox · group: slackbot

@AztecBot AztecBot added the claudebox Owned by claudebox. it can push to this PR. label May 22, 2026
@PhilWindle PhilWindle marked this pull request as ready for review May 22, 2026 15:43
@PhilWindle PhilWindle enabled auto-merge (squash) May 22, 2026 15:43
@PhilWindle PhilWindle disabled auto-merge May 22, 2026 16:02
@AztecBot
Copy link
Copy Markdown
Collaborator Author

Flakey Tests

🤖 says: This CI run detected 1 tests that failed, but were tolerated due to a .test_patterns.yml entry.

\033FLAKED\033 (8;;http://ci.aztec-labs.com/53f70755104db82b�53f70755104db82b8;;�):  yarn-project/end-to-end/scripts/run_test.sh simple src/e2e_cross_chain_messaging/l1_to_l2.parallel.test.ts "can send an L1 to L2 message from a non-registered portal address consumed from private repeatedly" (395s) (code: 0)

@PhilWindle PhilWindle merged commit 80fdec9 into merge-train/spartan May 23, 2026
32 of 37 checks passed
@PhilWindle PhilWindle deleted the cb/4bca97200b39 branch May 23, 2026 06:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

claudebox Owned by claudebox. it can push to this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants