Skip to content

Random share-safe titles never contain one another (flaky property test) - #519

Merged
mchestr merged 1 commit into
mainfrom
fix/share-safe-property-prefix
Nov 3, 2026
Merged

mchestr merged 1 commit into
mainfrom
fix/share-safe-property-prefix

Conversation

@mchestr

@mchestr mchestr commented Nov 3, 2026

Copy link
Copy Markdown
Owner

Why

lib/wrapped/data/__tests__/share-safe.property.test.ts failed on #514's CI run with fast-check seed -1167034155: the shrunk counterexample had a first title ZqtoLocaa and a last title ZqtoLoca. With only the last title hidden, the property's substring check (published.some((text) => text.includes(hidden))) matched the visible first title, so the test reported a leak that is not one. The generator draws each title fact independently from /^Zq[A-Za-z0-9 :'-]{4,60}$/, so one title being a prefix of another is a matter of luck, and the suite goes red once in a while on any PR.

What changes

The title dictionary rejects a set in which one title contains another (fc.filter); the property itself is unchanged. Reproduced the failure locally with the seed, confirmed the fix passes with that seed, and soaked it for 2,000 runs.

No product code changes.


Generated by Claude Code

The share-safe property hid a bookend title and asserted that no published
title or headline contains it. Two random titles can be a prefix of each
other ("ZqtoLocaa" and "ZqtoLoca", fast-check seed -1167034155), so a
visible first title carried the hidden last title by substring and the
property failed once in a while in CI. The generator now rejects a title
set in which one title contains another; the check itself is unchanged.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011TLtULGdZXTe6L61jNKqRF
@claude

claude Bot commented Nov 3, 2026 •

Copy link
Copy Markdown

Claude finished @mchestr's task in 13s —— View job


Review of PR #519

Verdict: looks good. It's a test-only change, and it fixes the flake as described.

Correctness

  • The check is values.every((title, i) => values.every((other, j) => i === j || !other.includes(title))). It is directional: no title may contain another as a substring. Two equal titles in different slots also fail it, because each contains the other. That is the right outcome here, since a hidden title equal to a visible one would also trip the property.
  • The filter applies to the whole titles dictionary, which is what the property's published.some(text => text.includes(hidden)) check needs. The series[1].title hidden case is covered too, because it is also a title fact.
  • The property is unchanged, so it still catches real leaks.

Notes (non-blocking)

  • The filter will reject very few cases. Titles are 6–62 characters drawn from a 66-character alphabet after the fixed Zq prefix, so a pair collides rarely. Without the fix the failure was also rare, which is how it slipped through, so a little rejection is expected. fast-check's default filter retry limit is not a concern here.
  • The generated titles are checked only against each other, not against the other strings the fixture publishes (the headline, other page titles). Those strings don't start with Zq, so no fix is needed there, but the Zq prefix is what keeps the test safe.
  • The doc comment explains the reason well, with the concrete ZqtoLocaa/ZqtoLoca example. The seed -1167034155 could go in the comment as well, or in a pinned examples/seed regression case, so the exact counterexample stays covered. This is optional.
  • I did not run the tests. CI covers them, and the PR description says the seed was reproduced and a 2,000-run soak passed.

CLAUDE.md conventions look fine: there is no any, no suppressions, and no product code is touched. No changelog fragment is needed.

@mchestr
mchestr merged commit be64fff into main Nov 3, 2026
22 checks passed
@mchestr
mchestr deleted the fix/share-safe-property-prefix branch November 3, 2026 15:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants