Skip to content

feat(standard-tests): check the cursor state of declarative sources in test_basic_read - #1270

Draft
Anatolii Yatsuk (tolik0) wants to merge 7 commits into
mainfrom
tolik0/cdk/per-partition-state-checks
Draft

Anatolii Yatsuk (tolik0) wants to merge 7 commits into
mainfrom
tolik0/cdk/per-partition-state-checks

Conversation

@tolik0

@tolik0 Anatolii Yatsuk (tolik0) commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

Resolves https://github.com/airbytehq/airbyte-internal-issues/issues/18410

What

test_basic_read now checks the cursor state that its read already emits, for a ConcurrentDeclarativeSource. It adds no extra read. The declarative factory builds the same cursor for a full refresh read as for an incremental sync, so this state is the one an incremental sync would emit.

Enforced checks (the test fails):

check what fails
checkpoints do not skip records emitted after them a record emitted after a state message whose resume would start past the record's cursor value, with no copy of it (same primary key, cursor value at least as high) emitted before that state. A record below the start of the slice that returned it counts from that start, because a resumed sync that requests the slice again reads it again (not under is_client_side_incremental, which drops records by value). A state past the cursor's end (a record dated in the future) bounds the resume at the end of its last slice, less the lookback window. A StateDelegatingStream is built from each state message to find the stream a resume would use; a state past api_retention_period reruns full_refresh_stream in full and flags nothing. A stream whose records cannot be matched to the read's slices is not checked (advisory note instead)
final state is not behind the newest record the final state of a ConcurrentCursor stream has no cursor value, one the cursor cannot parse, or one behind the newest record at the precision of the cursor's datetime_format
per-partition state is well formed use_global_cursor not true although the manifest sets global_substream_cursor, a states entry without a partition or cursor value, entries of the same parent stream with different partition keys (each parent of a multi-parent router may use its own partition_field), a global state without a cursor value, a lookback_window that is not a whole number of seconds between 0 and the read's duration
parent state is kept a stream with incremental_dependency has no parent_state for its parent
listed partitions have state a partition of a ListPartitionRouter has no states entry
state round trip the source cannot build its streams from its own final state: it raises, sends an HTTP request while building, or loads fewer partitions than the state holds

Advisory checks (CursorStateWarning, the test passes): records without a cursor value or with one the cursor cannot parse, and checks that could not run. Both found nothing in the fleet run. They stay advisory because the first depends on sandbox data (updated_at: null is normal) and the second points to a gap in the checker, not in the connector.

validate_state_messages: false in a basic_read entry turns these checks off, along with the state message check from 7.42.0. Such a config, and a config not listed under basic_read, reads without the hooks and with the production state throttle. ConcurrentPerPartitionCursor gets a STATE_EMIT_INTERVAL_SECONDS class attribute, so the suite can lift its 600 s state throttle for the first per-partition closes and check checkpoints there too. Failure messages give counts and field names, never record or state values, because CI logs are public.

Fleet dry run

184 connectors (177 manifest-only, 7 Python), against airbytehq/airbyte master ab1b47bce77, with this branch (d2a489d9) installed. 103 scenarios reached the checks; the rest already fail or skip test_basic_read on master (mostly expired credentials). No check crashed, and no test passed without the hooks but failed with them. 79 per-partition streams in 11 connectors ran with the lifted throttle.

check connectors flagged
checkpoints do not skip records emitted after them 1
every other check, enforced or advisory 0
connector stream(s) flagged records verdict
source-stripe customers, payment_intents, prices, plans, charges, invoices, products, setup_intents + 9 more 1,343 / 96 / 78 / 72 / 53 / 39 / 38 / 30 / ... real loss: resuming an interrupted first sync switches StateDelegatingStream to /events, which returns only the last 30 days

Compared with the first dry run (CDK at the first head of this PR, master fa95775cb59):

connector first run this run
source-lever-hiring fail on checkpoints (2 records) pass: the API returns records whose updatedAt is before updated_at_start, and the checker now counts a record from the start of its own slice
source-google-analytics-data-api fail on checkpoints (48 records) pass: yearWeek values are coarser than the read windows; same reason
source-calendly not in the population pass
source-aircall, source-slack fail before the checks pass
source-youtube-analytics pass fail: the playlist_*_a2 streams return no records, and the same without the hooks; the cursor checks found nothing
source-amazon-ads, source-nexus-datasets pass fail before the checks: 429 and "Dataset is not ready", the same without the hooks
source-fleetio, source-mntn, source-posthog pass timed out; see below

The GA4 run in the first dry run also exposed a real bug in that connector: yearWeek 202605 was parsed as 2026-05-01, which moves the state months ahead of the data in the first weeks of every year.

Not verified locally; these connectors run the checks in CI on release:

  • 20 manifest-only connectors with rotating refresh tokens, whose test secrets are single-use outside CI. Four of them have a per-partition stream with step, so the lifted checkpoint check applies: gitlab, pinterest, zendesk-support, zoom. Two use incremental_dependency: gong, pipedrive. Two use a ListPartitionRouter: greenhouse, trustpilot.
  • 10 connectors whose read timed out locally (2,700 s for manifest-only, 3,600 s for posthog): amplitude, coingecko-coins, devin-ai, fleetio, google-search-console, mailersend, microsoft-teams, mntn, posthog, rocket-chat. fleetio's sandbox has grown: its fuel_entries stream alone read more than 55,000 records in 9 minutes without the hooks. mntn and rocket-chat hang on retries (HTTP 500 and dropped connections; an SSL error on the sandbox host). posthog hangs on dropped connections in persons.
  • nasa and amazon-ads (rate limited locally), nexus-datasets (dataset not ready).
  • 4 Python sources that could not run locally: asana, freshcaller, google-ads, jina-ai-reader.

Slowest check: 5.2 s on a 277 s read (source-hubspot).

The review fixes in de7d1c5d only remove flags (retention fallbacks, multi-parent keys, unmatched slices), so this run still holds. Stripe's default config sets no api_retention_period, so it stays flagged.

Merge order

CI installs the CDK unpinned, and a CDK release ships every PR merged to main, so this PR stays in draft until this connector PR merges:

Before this PR is marked ready, a pre-release of this head runs connector CI on the 34 connectors the dry run did not cover (above), starting with gitlab, pinterest, zendesk-support and zoom, where the lifted checkpoint check applies.

lever-hiring and GA4 now pass without an opt-out, so airbytehq/airbyte#91056 (lever-hiring opt-out) and the opt-out in airbytehq/airbyte#91057 no longer gate this PR. The GA4 yearWeek/yearMonth parsing fix in #91057 stands on its own.

Conflicts with open CDK PRs

git merge-tree of this head against each PR's head, leaving out conflicts they already have with main:

No order is required; the conflicts are textual and the second PR to merge resolves them.

Testing

  • unit_tests/test/test_cursor_state_checks.py: 83 tests; unit_tests/test: 1221 passed. The checks run without network access in the tests (_no_http).
  • Mutation pass: 70 mutations in the first round (68 killed, 2 equivalent: a deepcopy of a state that is loaded only once, and >= vs > when pushing equal bounds onto a monotone stack), then 19 for the review fixes (rows 70-88: 18 killed, 1 equivalent). Two guards that survived were removed rather than kept: a redundant check_checkpoints = False (the stream has no slices left to check) and a parse guard for a slice start the cursor formatted itself. Each was applied alone to a scratch copy and run against the test file. Rows 11, 12, 25, 26, 35, 36 and 62 were re-run against the current code. Rows 89-104 cover the second review round; rows 30 and 31 no longer apply, because the final-state cursor template they mutated was removed (only StateDelegatingStream needed it, and it now builds the stream per state message).
Mutations
# mutation result failing tests
0 recorder: install returns None for non-CDS -> hooks anything KILLED test_other_sources_are_not_checked
1 uninstall: keep streams hook KILLED test_basic_read_leaves_no_hooks_on_the_source
2 uninstall: keep _handle_item hook KILLED test_basic_read_leaves_no_hooks_on_the_source
3 lift only with step: always lift KILLED test_checkpoint_check_does_not_flag_a_per_partition_stream_without_step
4 lift only with step: never lift KILLED test_checkpoint_check_flags_a_per_partition_stream_with_step, test_throttle_lift_emits_more_per_partition_states_within_budget, test_lifted_closes_thin_out_after_the_first_ones
5 _has_step: ignore timedelta.max KILLED test_checkpoint_check_does_not_flag_a_per_partition_stream_without_step
6 throttle: never lift (interval stays default) KILLED test_throttle_lift_emits_more_per_partition_states_within_budget, test_lifted_closes_thin_out_after_the_first_ones
7 throttle: always lift (no budget) KILLED test_throttle_lift_emits_more_per_partition_states_within_budget, test_lifted_closes_thin_out_after_the_first_ones
8 throttle: drop MAX_LIFTED_CLOSES KILLED test_throttle_lift_emits_more_per_partition_states_within_budget
9 throttle: drop FIRST closes KILLED test_throttle_lift_emits_more_per_partition_states_within_budget
10 throttle: drop growth KILLED test_lifted_closes_thin_out_after_the_first_ones
11 _handle: do not record slices KILLED test_checkpoint_check_flags_a_per_partition_stream_with_step, test_per_partition_records_below_their_slice_start_are_not_flagged, test_records_below_their_slice_start_are_not_flagged, test_unmatched_partitions_skip_the_per_partition_checkpoint_check
12 _slice_of: partition None KILLED test_checkpoint_check_flags_a_per_partition_stream_with_step
13 _resume_start: raw state instead of slice start KILLED test_checkpoint_bound_comes_from_the_resumed_cursor, test_checkpoint_bound_of_a_state_delegating_stream_comes_from_its_incremental_stream
14 _sub_cursor: pass runtime lookback 0 -> original state no deepcopy SURVIVED
15 bound cap: no fallback to earlier bound KILLED test_bounds_past_the_cap_use_the_bound_of_an_earlier_state
16 bound cap: ignore cap KILLED test_bounds_past_the_cap_never_flag_more
17 monotone push: push every state KILLED test_a_state_that_moves_back_does_not_lower_the_bound
18 monotone push: >= instead of > SURVIVED
19 checkpoint flag: value <= bound KILLED test_a_record_at_the_resume_start_is_not_flagged
20 dedupe: no copies exemption KILLED test_checkpoint_check_does_not_flag_overlapping_slices[api_ignores_until], test_checkpoint_check_does_not_flag_overlapping_slices[no_end_option], test_checkpoint_check_compares_whole_records_without_a_primary_key
21 dedupe: copy value > instead of >= KILLED test_checkpoint_check_does_not_flag_overlapping_slices[api_ignores_until], test_checkpoint_check_does_not_flag_overlapping_slices[no_end_option], test_checkpoint_check_compares_whole_records_without_a_primary_key
22 dedupe: ignore copy position KILLED test_a_state_that_moves_back_does_not_lower_the_bound, test_a_copy_after_the_first_skipping_state_does_not_exempt_a_record
23 binary search: use last checkpoint KILLED test_a_copy_after_the_first_skipping_state_does_not_exempt_a_record
24 copy_key: no-PK uses empty list KILLED test_checkpoint_check_compares_whole_records_without_a_primary_key
25 slice count mismatch: keep the slices KILLED test_records_that_do_not_match_the_read_are_checked_by_their_own_value
26 partitioned slice count mismatch: not reported KILLED test_unmatched_partitions_skip_the_per_partition_checkpoint_check
27 scope: per-partition checkpoints always KILLED test_checkpoint_check_does_not_flag_a_per_partition_stream_without_step, test_suites_whose_read_builds_another_source_get_the_concurrent_cursor_part, test_final_state_check_skips_per_partition_streams
28 scope: per-partition checkpoints without lift KILLED test_checkpoint_check_does_not_flag_a_per_partition_stream_without_step, test_final_state_check_skips_per_partition_streams, test_lookback_window_up_to_the_read_duration_rounded_up_passes
29 scope: isinstance instead of type for ConcurrentCursor KILLED test_custom_cursor_subclasses_are_skipped
30 template: never from rebuilt KILLED test_checkpoint_bound_of_a_state_delegating_stream_comes_from_its_incremental_stream
31 template: any rebuilt cursor KILLED test_a_rebuilt_stream_without_a_concurrent_cursor_keeps_the_read_cursor
32 rebuild: no per-stream fallback KILLED test_a_final_state_the_source_cannot_load_fails, test_a_build_that_sends_http_fails
33 rebuild: no _no_http KILLED test_a_build_that_sends_http_fails
34 rebuild: HTTP reported as generic KILLED test_a_build_that_sends_http_fails
35 record safety net removed KILLED test_a_check_that_crashes_is_reported_as_advisory[add_record], test_a_crash_in_the_checkpoint_walk_keeps_the_other_checks[_resume_start]
36 state safety net removed KILLED test_a_check_that_crashes_is_reported_as_advisory[add_state], test_a_crash_in_the_checkpoint_walk_keeps_the_other_checks[_add_checkpoint]
37 report safety net removed KILLED test_a_check_that_crashes_is_reported_as_advisory[add_checkpoint_report]
38 final state: no precision formatting KILLED test_final_state_is_compared_at_the_state_precision
39 final state: <= instead of < KILLED test_checkpoint_check_flags_records_past_their_slice, test_checkpoint_check_passes_a_correct_api, test_checkpoint_check_does_not_flag_overlapping_slices[api_ignores_until]
40 final state: never flags behind KILLED test_final_state_behind_the_newest_record_fails
41 final state: missing key not flagged KILLED test_final_state_without_the_cursor_field_fails
42 final state: unparseable not flagged KILLED test_final_state_the_cursor_cannot_parse_fails
43 final state: also for partitioned KILLED test_checkpoint_check_flags_a_per_partition_stream_with_step, test_checkpoint_check_does_not_flag_a_per_partition_stream_without_step, test_suites_whose_read_builds_another_source_get_the_concurrent_cursor_part
44 cursor values: missing not counted KILLED test_missing_and_unparseable_cursor_values_are_advisory, test_global_state_is_not_required_when_no_record_has_a_cursor_value
45 cursor values: unparseable not counted KILLED test_missing_and_unparseable_cursor_values_are_advisory
46 cursor values: enforced KILLED test_missing_and_unparseable_cursor_values_are_advisory, test_basic_read_warns_on_an_advisory_cursor_state_problem
47 shape: global_at_start ignored KILLED test_global_substream_cursor_must_be_used
48 shape: entry without partition ignored KILLED test_malformed_per_partition_state_fails[entry_without_partition]
49 shape: entry without cursor ignored KILLED test_malformed_per_partition_state_fails[entry_without_cursor]
50 shape: empty partition allowed KILLED test_malformed_per_partition_state_fails[entry_without_partition]
51 shape: key sets ignored KILLED test_malformed_per_partition_state_fails[different_partition_keys]
52 shape: global state ignored KILLED test_global_state_must_hold_the_cursor
53 shape: global state checked without values KILLED test_global_state_is_not_required_when_no_record_has_a_cursor_value
54 shape: bool lookback allowed KILLED test_malformed_per_partition_state_fails[bool]
55 shape: lookback upper bound ignored KILLED test_malformed_per_partition_state_fails[lookback_longer_than_the_read], test_lookback_window_up_to_the_read_duration_rounded_up_passes
56 shape: lookback lower bound ignored KILLED test_malformed_per_partition_state_fails[negative]
57 shape: lookback floor instead of ceil KILLED test_lookback_window_up_to_the_read_duration_rounded_up_passes
58 parent: never flagged KILLED test_missing_parent_state_fails, test_parent_state_is_required_behind_a_grouping_router
59 parent: incremental_dependency ignored KILLED test_parent_state_is_not_required_without_incremental_dependency
60 parent: grouping not unwrapped KILLED test_parent_state_is_required_behind_a_grouping_router
61 coverage: never flagged KILLED test_listed_partition_without_state_fails
62 coverage: any router KILLED test_partition_coverage_is_only_checked_for_a_list_router, test_suites_whose_read_builds_another_source_get_the_concurrent_cursor_part
63 round trip: count never compared KILLED test_a_cursor_that_drops_partitions_on_load_fails
64 round trip: build errors not reported KILLED test_a_final_state_the_source_cannot_load_fails, test_a_build_that_sends_http_fails
65 suite: opt-out ignored KILLED test_basic_read_fails_on_an_enforced_cursor_state_problem
66 suite: advisory not warned KILLED test_basic_read_warns_on_an_advisory_cursor_state_problem
67 suite: enforced not asserted KILLED test_basic_read_fails_on_an_enforced_cursor_state_problem
68 suite: no uninstall KILLED test_basic_read_leaves_no_hooks_on_the_source
69 suite: recorder not passed KILLED test_basic_read_fails_on_an_enforced_cursor_state_problem, test_basic_read_warns_on_an_advisory_cursor_state_problem
70 _resume_start: None when the resumed cursor has no slice KILLED test_a_future_dated_record_does_not_silence_the_checkpoint_check[0], [5], test_a_resume_past_the_end_starts_a_lookback_window_before_the_state
71 past-end bound: no lookback window KILLED test_a_resume_past_the_end_starts_a_lookback_window_before_the_state
72 no slice-start exemption KILLED test_records_below_their_slice_start_are_not_flagged, test_per_partition_records_below_their_slice_start_are_not_flagged
73 slice-start exemption also under is_client_side_incremental KILLED test_records_below_their_slice_start_are_flagged_under_client_side_incremental, test_client_side_incremental_comes_from_the_incremental_stream_of_a_state_delegating_stream
74 client-side incremental streams: none found KILLED same two tests
75 client side read from the top-level definition only KILLED test_client_side_incremental_comes_from_the_incremental_stream_of_a_state_delegating_stream
76 partition taken from the slice for unpartitioned streams KILLED 11 tests, e.g. test_checkpoint_check_flags_records_past_their_slice
77 slice cache reused across slices KILLED test_records_below_their_slice_start_are_not_flagged, test_per_partition_records_below_their_slice_start_are_not_flagged, test_basic_read_bounds_the_lookback_window_by_the_read_duration
78 _slice_of: no slice start KILLED test_records_below_their_slice_start_are_not_flagged, test_per_partition_records_below_their_slice_start_are_not_flagged
79 a crash drops the stream from every check KILLED test_a_crash_in_the_checkpoint_walk_keeps_the_other_checks[_resume_start], [_add_checkpoint]
80 add_state keeps collecting checkpoints after a crash KILLED test_a_crash_in_the_checkpoint_walk_keeps_the_other_checks[_add_checkpoint]
81 suite: hooks installed whatever the scenario KILLED test_basic_read_installs_no_hooks_when_it_does_not_check_the_cursor_state[opted_out], [not_a_basic_read_config]
82 suite: hooks installed when opted out KILLED test_basic_read_installs_no_hooks_when_it_does_not_check_the_cursor_state[opted_out]
83 suite: reference_source=None KILLED test_basic_read_checks_a_suite_whose_read_builds_another_source, test_basic_read_warns_when_the_reference_source_cannot_build
84 suite: read_seconds = 0.0 KILLED test_basic_read_bounds_the_lookback_window_by_the_read_duration, test_basic_read_checks_a_suite_whose_read_builds_another_source
85 suite: uninstall only after a successful read KILLED test_basic_read_removes_its_hooks_when_the_read_fails
86 reference path: global_substream_cursor flags dropped KILLED test_basic_read_checks_a_suite_whose_read_builds_another_source
87 reference path: build error returns no problems KILLED test_basic_read_warns_when_the_reference_source_cannot_build
88 slice start compared with >= SURVIVED equivalent: on a tie both give the record's value
89 StateDelegatingStream: bound from the final-state stream (no per-checkpoint build) KILLED test_checkpoint_bound_of_a_state_delegating_stream_follows_its_retention_period[states_past_retention-*], test_rebuilding_a_state_delegating_stream_emits_nothing_on_the_read_queue, test_a_resume_without_a_concurrent_cursor_reads_in_full, test_rebuilds_of_a_state_delegating_stream_past_the_cap_never_flag_more
90 bound ignores a cleared state KILLED ...follows_its_retention_period[states_past_retention-per_partition]
91 build never reports a cleared state KILLED same
92 round trip compares a rebuild that cleared the state KILLED same
93 build emits on the read's queue (no repository swap) KILLED test_rebuilding_a_state_delegating_stream_emits_nothing_on_the_read_queue
94 no cap on builds KILLED test_rebuilds_of_a_state_delegating_stream_past_the_cap_never_flag_more
95 shape: any two key sets differ KILLED test_partition_keys_are_compared_per_parent[one_key_set_per_parent]
96 shape: entries without a parent field ignored KILLED test_malformed_per_partition_state_fails[different_partition_keys], test_partition_keys_are_compared_per_parent[no_parent_field]
97 shape: differing keys of one parent ignored KILLED test_partition_keys_are_compared_per_parent[different_keys_for_one_parent]
98 unmatched slices: unpartitioned stream checked by value KILLED test_records_that_do_not_match_the_read_skip_the_checkpoint_check, test_suites_whose_read_builds_another_source_skip_the_checkpoint_check
99 read on another source: checked by value KILLED test_suites_whose_read_builds_another_source_skip_the_checkpoint_check
100 StateDelegatingStream bounds shared by equal partition states KILLED test_equal_partition_states_in_different_state_messages_resume_apart
101 per-partition resume ignores the built cursor KILLED ...follows_its_retention_period[no_retention-per_partition]
102 resumed stream without a concurrent cursor bounded by the read's cursor KILLED test_a_resume_without_a_concurrent_cursor_reads_in_full
103 no cap on StateDelegatingStream bounds KILLED test_bounds_past_the_cap_never_flag_more[state_delegating]
104 final-state cursor template kept (removed code) n/a 83 tests pass with or without it, so it was removed

Not checked

Streams without a ConcurrentCursor or ConcurrentPerPartitionCursor (no cursor, FinalStateCursor, custom or file-based cursors) are skipped, never failed. Sources that are not a ConcurrentDeclarativeSource are not checked. The Docker read (test_docker_image_build_and_read) is not checked either, because the checks need the source object in-process to rebuild cursors. This deviates from the issue, which expects the Docker read to get the ConcurrentCursor checks.

🤖 Generated with Claude Code

…n test_basic_read

test_basic_read now checks the cursor state the read already emits for a
ConcurrentDeclarativeSource: checkpoints that a resume would skip records
from, a final state behind the newest record, per-partition state shape,
parent state, partition coverage and a round trip through the cursor.
Cursor values and check errors are reported as CursorStateWarning.
`validate_state_messages: false` turns the checks off.
@github-actions

github-actions Bot commented Nov 5, 2026

Copy link
Copy Markdown

👋 Greetings, Airbyte Team Member!

Here are some helpful tips and reminders for your convenience.

💡 Show Tips and Tricks

Testing This CDK Version

You can test this version of the CDK using the following:

# Run the CLI from this branch:
uvx 'git+https://github.com/airbytehq/airbyte-python-cdk.git@tolik0/cdk/per-partition-state-checks#egg=airbyte-python-cdk[dev]' --help

# Update a connector to use the CDK from this branch ref:
cd airbyte-integrations/connectors/source-example
poe use-cdk-branch tolik0/cdk/per-partition-state-checks

PR Slash Commands

Airbyte Maintainers can execute the following slash commands on your PR:

  • /autofix - Fixes most formatting and linting issues
  • /poetry-lock - Updates poetry.lock file
  • /test - Runs connector tests with the updated CDK
  • /prerelease - Triggers a prerelease publish with default arguments
  • /poe build - Regenerate git-committed build artifacts, such as the pydantic models which are generated from the manifest JSON schema in YAML.
  • /poe <command> - Runs any poe command in the CDK environment
📚 Show Repo Guidance

Helpful Resources

📝 Edit this welcome message.

@github-actions

github-actions Bot commented Nov 5, 2026 •

Copy link
Copy Markdown

PyTest Results (Fast)

5 306 tests   - 1 128   5 269 ✅  - 1 128   6m 41s ⏱️ +14s
    1 suites ±    0      36 💤  -     1 
    1 files   ±    0       1 ❌ +    1 

For more details on these failures, see this check.

Results for commit de7d1c5. ± Comparison against base commit 314f8fb.

This pull request removes 1206 and adds 78 tests. Note that renamed tests count towards both.
unit_tests.test.mock_http.test_matcher.HttpRequestMatcherTest ‑ test_given_expected_number_of_requests_met_when_matches_then_has_expected_match_count
unit_tests.test.mock_http.test_matcher.HttpRequestMatcherTest ‑ test_given_expected_number_of_requests_not_met_when_matches_then_does_not_have_expected_match_count
unit_tests.test.mock_http.test_matcher.HttpRequestMatcherTest ‑ test_given_many_requests_with_some_match_when_matches_then_has_expected_match_count
unit_tests.test.mock_http.test_matcher.HttpRequestMatcherTest ‑ test_given_request_does_not_match_when_matches_then_does_not_have_expected_match_count
unit_tests.test.mock_http.test_matcher.HttpRequestMatcherTest ‑ test_given_request_matches_when_matches_then_has_expected_match_count
unit_tests.test.mock_http.test_mocker.HttpMockerTest ‑ test_given_all_requests_match_when_decorate_then_do_not_raise
unit_tests.test.mock_http.test_mocker.HttpMockerTest ‑ test_given_assertion_error_but_missing_request_when_decorate_then_raise_missing_http_request
unit_tests.test.mock_http.test_mocker.HttpMockerTest ‑ test_given_assertion_error_when_decorate_then_raise_assertion_error
unit_tests.test.mock_http.test_mocker.HttpMockerTest ‑ test_given_delete_request_match_when_decorate_then_return_response
unit_tests.test.mock_http.test_mocker.HttpMockerTest ‑ test_given_exact_number_of_call_provided_when_assert_number_of_calls_then_do_not_raise
…
unit_tests.sources.declarative.async_job.test_job_orchestrator.AsyncJobOrchestratorTest ‑ test_given_exception_when_start_job_then_emitted_trace_masks_config_secrets
unit_tests.sources.declarative.auth.test_jwt ‑ test_iat_backdate_moves_iat_and_nbf_but_not_exp[backdated_by_60s]
unit_tests.sources.declarative.auth.test_jwt ‑ test_iat_backdate_moves_iat_and_nbf_but_not_exp[default_is_now]
unit_tests.sources.declarative.auth.test_jwt ‑ test_iat_backdate_moves_iat_and_nbf_but_not_exp[zero_is_now]
unit_tests.sources.declarative.auth.test_jwt ‑ test_iat_backdate_seconds_is_normalized[none_defaults_to_zero]
unit_tests.sources.declarative.auth.test_jwt ‑ test_iat_backdate_seconds_is_normalized[positive_is_kept]
unit_tests.sources.declarative.auth.test_jwt ‑ test_iss_sub_and_kid_from_config_stay_strings[boolean_like]
unit_tests.sources.declarative.auth.test_jwt ‑ test_iss_sub_and_kid_from_config_stay_strings[float_like]
unit_tests.sources.declarative.auth.test_jwt ‑ test_iss_sub_and_kid_from_config_stay_strings[integer_like]
unit_tests.sources.declarative.auth.test_jwt ‑ test_iss_sub_and_kid_from_config_stay_strings[plain_string]
…

♻️ This comment has been updated with latest results.

@github-actions

github-actions Bot commented Nov 5, 2026 •

Copy link
Copy Markdown

PyTest Results (Full)

6 580 tests  +143   6 544 ✅ +144   11m 57s ⏱️ +21s
    1 suites ±  0      36 💤  -   1 
    1 files   ±  0       0 ❌ ±  0 

Results for commit de7d1c5. ± Comparison against base commit 314f8fb.

♻️ This comment has been updated with latest results.

…ion-state-checks

# Conflicts:
#	airbyte_cdk/test/standard_tests/source_base.py
…or state checks

- A state past the cursor's end (a record dated in the future) no longer turns the
  checkpoint check off: a cursor with no slice resumes from the end of its last state
  slice, less the lookback window.
- A record below the start of the slice that returned it counts from that start, since a
  resumed sync that requests the slice again reads it again. Streams with
  `is_client_side_incremental` still compare the record's own value.
- `test_basic_read` installs the hooks only on a read whose cursor state it checks.
- A crash while walking a stream's records or states stops only that stream's checkpoint
  check; the final-state, shape, parent, coverage and round-trip checks still run.
- Tests pin the suite wiring (read duration, reference source, hook removal after a failed
  read) and run the checks without network access.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes for one soundness issue in the checkpoint model (inline, 🔴), plus three medium notes (🟠).

What's done well: no second read, resume starts computed with the CDK's own cursor code (gaps, lookback, clamping all come out right), failure messages carry no values, and the tests are real end-to-end reads. I collected and ran the 71 tests, re-applied mutations #19, #72 and #59 (all killed), and the survivors #14, #18 and #88 are equivalent. Reverting the throttle to the literal 600 fails two tests, so the production refactor is covered and behaviour-identical. No runtime, Builder, SDM or manifest-server impact; feat: matches the 7.36/7.40/7.42 precedent.

Rollout: Check: destination-motherduck is red on #1266–#1273 as well, so it's infra. Merge order with airbytehq/airbyte#91061 is as described.

if isinstance(resumed, (ConcurrentCursor, ConcurrentPerPartitionCursor)):
# A resumed sync builds the stream from its state: a `StateDelegatingStream` then
# builds its `incremental_stream`, so its cursor sets the resume start.
stream_check.template = resumed

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 High: a StateDelegatingStream resume is modelled with the stream built from the final state, for every checkpoint.

create_state_delegating_stream picks the stream per state, but this template is built once from the final state and then used for every checkpoint's bound (L472-L476). Two cases where that gives a false positive:

Suggested fix: pick the template per checkpoint. Either rebuild with each checkpoint's state (_build_with_states already exists; cache by state key under the 2,000 cap), or return None as the bound when that state would select full_refresh_stream. The retention case would make a good passing test.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in de7d1c5: a StateDelegatingStream now builds the stream from each state message to get that checkpoint's resume start (at most 200 builds per stream; past that a state uses an earlier, lower bound). When the build clears the state (past api_retention_period), the resume reads everything, so nothing is flagged. The same fix applies to the round-trip partition count, which flagged 0 partition(s) for a per-partition stream past retention. The build no longer emits its cleared state onto the read's queue. Tests: test_checkpoint_bound_of_a_state_delegating_stream_follows_its_retention_period (P30D passes; with no retention it's flagged; concurrent and per-partition), test_states_within_the_retention_period_resume_incrementally, test_rebuilding_a_state_delegating_stream_emits_nothing_on_the_read_queue, test_rebuilds_of_a_state_delegating_stream_past_the_cap_never_flag_more. After #1256, an in-progress state that selects full_refresh_stream gets that stream's cursor, so the Stripe opt-out can come off.

),
)
enforced = {check: found for check, found in cursor_problems.items() if check.enforced}
assert not enforced, (

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟠 Medium (rollout): these checks are enforced from the first release.

With the CLI unpinned in the monorepo, about 204 manifest-only connectors with a basic_read entry get these checks on their next CI run. The 34 that the dry run couldn't cover will find out from a red build on an unrelated PR. That includes the 20 with rotating refresh tokens, and four of those (gitlab, pinterest, zendesk-support, zoom) hit the lifted checkpoint check. 7.42.0 shipped stream statuses as a warning first.

Either of these would close the gap:

  • ship as CursorStateWarning for one release, then enforce;
  • before merge, run a pre-release plus monorepo CI on those connectors.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Keeping the checks enforced. I'm publishing a pre-release of de7d1c5 with /prerelease and will run connector CI against it on the 34 connectors the dry run didn't cover, starting with gitlab, pinterest, zendesk-support and zoom. The PR stays in draft until they pass or get a fix or an opt-out PR that merges first.

for entry in entries
if isinstance(entry, Mapping) and isinstance(entry.get("partition"), Mapping)
}
if len(key_sets) > 1:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟠 Medium: false positive on a valid multi-parent router.

A SubstreamPartitionRouter with two parent_stream_configs using different partition_field values (parent_id and other_id) against a healthy API fails the enforced PARTITION_STATE_SHAPE with "states entries use different partition keys (other_id, parent_slice; parent_id, parent_slice)". I confirmed it with a probe. No monorepo connector does this today: the 4 multi-parent routers (asana, discord, pipedrive, stripe) all use one field.

Suggested fix: compare key sets per parent config, or make this item advisory, and add a passing test with the two-parent manifest.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in de7d1c5: entries are grouped by the parent partition_field they carry, and only entries of the same parent (or entries with none of the router's fields) must use the same keys. test_partition_keys_are_compared_per_parent reads a two-parent router (parent_id, other_id) and passes; it still fails when one parent's entries differ or an entry has neither field.

# Without slices, a record's own value is where a resume reads it again. That can
# only flag more, so a stream without partitions still gets its checkpoints checked.
# A stream with partitions cannot tell which partition's checkpoints apply.
stream_check.slices = None

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟠 Medium: the fallback can raise more false alarms.

"That can only flag more" means it can only add false positives. Without slices, the slice-start exemption that fixed lever-hiring and GA4 is gone. test_records_that_do_not_match_the_read_are_checked_by_their_own_value shows it: the API is healthy and the test still asserts [CHECKPOINTS]. On the normal path the counts match, so this is a candidate only.

Suggested fix: do what the partitioned branch does. Skip the checkpoint check for that stream and report it under the advisory CHECK_ERRORS.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in de7d1c5: a stream whose records can't be matched to the read's slices skips the checkpoint check and reports it under CHECK_ERRORS, partitioned or not. The same applies when the read ran on another source instance (reference_source): the slices were not recorded, so a ConcurrentCursor stream with step now skips the check with a CHECK_ERRORS line instead of checking by value.

…h the stream its state selects

- Build a `StateDelegatingStream` from each state message to get that checkpoint's resume
  start, instead of from the final state. A state past `api_retention_period` reruns
  `full_refresh_stream` in full, so it flags nothing. Builds are capped at 200 per stream and
  no longer put their cleared state on the read's queue.
- Skip the round-trip partition count when the build from the final state cleared it.
- Compare partition keys per parent `partition_field`, so a router with several parents that
  use different fields passes.
- Skip the checkpoint check, with an advisory `CHECK_ERRORS` line, when a stream's records
  cannot be matched to the read's slices or the read ran on another source instance, instead
  of checking each record from its own value.
- Drop the final-state cursor template, which only `StateDelegatingStream` needed.
- Tests: monkeypatch the checks module by its dotted path instead of importing it twice.
@tolik0

Anatolii Yatsuk (tolik0) commented Nov 10, 2026 •

Copy link
Copy Markdown
Contributor Author

/prerelease

Prerelease Job Info

This job triggers the publish workflow with default arguments to create a prerelease.

Prerelease job started... Check job output.

✅ Prerelease workflow triggered successfully.

View the publish workflow run: https://github.com/airbytehq/airbyte-python-cdk/actions/runs/42871066020

This branch was successfully deployed

2 active deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants