Skip to content

probation_start is set while the criterion is still failing #1844

Description

@cka-y

probation_start is stamped as soon as a failure is confirmed, then re-stamped to "tomorrow" on every failing night after that. So a set probation_start does not mean the criterion is serving probation — it may still be failing.

Every reader has to compensate with the same extra check:

Reader Compensation
is_serving_probation (API) confirmed_status == 'pass'
feed_search.sql confirmed_status = 'pass'
roll_up_on_probation cancels on any confirmed failure
state_machine.phase() returns ON_PROBATION for both, leaves it to the caller

Recent fixes have all been on the read side: #1840, #1841, #1842, #1843.

Fix

Stamp probation_start only on the recovery — the run where confirmed_status goes fail → pass. last_confirmed_failure_at already records that a failure happened, so the "stamp tomorrow on every failing night" trick is no longer needed to land the start on the repair day.

Then probation_start is not None means serving probation, and the four compensations above come out.

  • state_machine._probation_start, plus the outlived_grace branch in transition(), which stamps too
  • backfill for rows currently stamped mid-failure

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions