[Anatolii Yatsuk (tolik0)]

Part of airbytehq/airbyte-internal-issues#17137 (P1.4 "Docker read test", under airbytehq/airbyte-internal-issues#17129). Companion monorepo PR: airbytehq/airbyte#88939.

The Docker read does not change on release. It stays off by default, as on main. This PR makes it check what it reads when it is turned on. One change does reach every connector: its Docker check and discover tests run in a writable /source working directory (item 4). airbytehq/airbyte#88939 then turns it on in connector CI, but only for Python sources whose pinned CDK is too old for the in-process standard tests. Those sources get no CI read today.

What

test_docker_image_build_and_read runs read inside the connector image. It is skipped by default, and when it was turned on:

• It read only the connector's suggestedStreams, when the connector declared them.
• The records were not checked.
• A Python connector could not write to its working directory.
• --read-scenarios skipped every read of a connector whose configs had other names.

(The empty_streams crash that #17137 also describes was fixed in 7.36.0 by #1142.)

Since airbytehq/airbyte-internal-issues#17133, connector CI reads every connector whose in-process tests run, through test_basic_read. A Docker read on by default would read each of them a second time. The decision is recorded in #17137. So the read stays opt-in, and this PR fixes it:

1. Off by default, consistently. --read-from-streams defaults to false. The addoption default, the fixture fallback and the docstring agree. default turns the read on. true, all and suggested are aliases of default, so existing commands still run. A comma list of stream names narrows the read.
2. Same streams as test_basic_read. The read covers the config's configured_catalog_path, or every discovered stream, minus empty_streams (get_read_streams, shared with test_basic_read). The connector's suggestedStreams no longer narrow it: they are what the UI preselects, not a test selection.
3. Same checks as test_basic_read. A config under basic_read goes through the same assert_read_records call, with the same arguments:
   • Every stream it reads must return records.
   • Every record must match its stream's schema. validate_schema: false opts out, and is rejected at test_strictness_level: high before any Docker command.
   • Records have usable primary keys (feat(standard-tests): check primary keys, state messages and error traces in read tests, and report stream statuses #1230). validate_primary_keys_data_type: false opts out.
   • Every stream read emits a per-stream state message with sourceStats (feat(standard-tests): check primary keys, state messages and error traces in read tests, and report stream statuses #1230). validate_state_messages: false opts out.
   • Incomplete stream statuses raise the StreamStatusWarning (feat(standard-tests): check primary keys, state messages and error traces in read tests, and report stream statuses #1230), not a failure.
   • A config only under full_refresh must return some records.
   • Failure messages name streams, record numbers and schema paths, never record or config values.
4. A writable /source working directory (an in-memory tmpfs) for the Docker check, discover and read. The platform uses /source for a source's read, and /config for check and discover; any writable directory outside the image serves those tests. A Python connector image's WORKDIR is not writable by the airbyte user, so source-shopify and source-salesforce failed with a PermissionError on bulk-export temp files that a sync never hits.
5. --read-scenarios fixes.
   • The default reads configs named config, valid_config or default, and otherwise the first basic_read config (or the first read config). Before, 12 source connectors with other config names skipped every read.
   • When none of the picked configs is listed under basic_read, the default also reads the first basic_read config, so the read gets the basic_read checks. source-intercom, whose config is only under full_refresh, skipped them.
   • none and lists skip instead of raising a ValueError.
   • A value of either option that names nothing (--read-from-streams=, --read-scenarios=,) is a usage error, so an empty CI variable cannot turn the read off with a green run.
   • A misspelled name fails instead of skipping every read: a --read-scenarios name that is not one of the connector's read configs, or a --read-from-streams name that discover does not report for the config. Stream names match exactly, case included. A config whose catalog or empty_streams leaves out every listed, discovered stream is still skipped.
6. The CDK's connector matrix turns the read on in its image test step (PYTEST_ADDOPTS: --read-from-streams=default), so the read path keeps running in CI.

How

Option	Default	Other values
--read-from-streams (whether to read, which streams)	false: skipped	default (also true, all, suggested): the streams test_basic_read reads; a comma list of stream names narrows them (exact names; one that discover does not report fails); none also skips (CDK 7.42.0 and earlier read none as a stream name, so use false); an empty value is an error
--read-scenarios (which configs, once the read is on)	default: configs named config, valid_config or default, plus the first basic_read config when none of those is one; else the first basic_read (or read) config expected to succeed	all, a comma list of config names (one that is not a read config fails), none (skip)

The test calls assert_read_records, assert_schema_validation_opt_out_allowed and find_stream_status_problems from _read_assertions.py, as test_basic_read does, so a check added there reaches both reads through the same call.

Files

• airbyte_cdk/test/standard_tests/pytest_hooks.py: false default for --read-from-streams, fixture parsing, docstrings.
• airbyte_cdk/test/standard_tests/docker_base.py: default scenario selection, stream selection without suggestedStreams, record, primary key, state and stream status checks, /source working directory for check, discover and read.
• unit_tests/test/test_docker_read.py (new).
• unit_tests/test/standard_test_stubs.py (new): the scenario, suite and Docker stubs that test_scenario_sections.py and test_docker_read.py share, moved out of test_scenario_sections.py. The read stubs emit state messages. run_without_skip fails a unit test whose directly called standard test method skips; both files use it for every call that must run.
• unit_tests/test/test_check_assertions.py: the Docker check tests stub spec, so they run instead of skipping on the secret leak check (they all skipped since feat(standard-tests): lint airbyte_secret marking and secret leaks #1219), and assert the working directory.
• .github/workflows/connector-tests.yml: the image test step turns the Docker read on.
• cdk-migrations.md: new "Upgrading to the next minor release (unreleased)" section. main's unreleased section for feat(standard-tests): check primary keys, state messages and error traces in read tests, and report stream statuses #1230 is renamed to "Upgrading to 7.42.0", the release that shipped it.

Cost of the read, when it runs

Measured on the earlier revision of this branch, which narrowed the read to suggestedStreams, against the published images of airbytehq/airbyte master:

• Python sources: median 24 s, p90 85 s, 229 s for shopify and facebook-marketing.
• Marketo: 34 minutes. It declares no suggestedStreams, so it read every activities_* export.
• 5 certified manifest-only connectors: 11 to 159 s.

This is the cost a default-on read would have added to every connector that already reads in process, and why the read stays opt-in. Dropping the suggestedStreams narrowing changes the read only for connectors that declare them. Of the Python sources with a CDK too old for the in-process tests, which are the ones airbytehq/airbyte#88939 will read, only source-mixpanel does.

Rollout

• Minor release, no warn-only period. Nothing changes for a connector's read unless its CI turns it on. The Docker check and discover move to /source for every connector (item 4).
• ci(connectors): read old-CDK Python sources from their image airbyte#88939 merges after this release. Before it, --read-from-streams=default on a released CLI would read only suggestedStreams where declared, check no records, and fail on a Python connector that writes to its working directory. Its fleet dry run of the old-CDK Python sources runs against this release.
• feat(standard-tests): check primary keys, state messages and error traces in read tests, and report stream statuses #1230 (P1.2 slice 2) landed first, in 7.42.0. This PR passes its checks to the Docker read too. A connector that turns the read on is then held to 7.42.0's test_basic_read checks.

Test plan

• unit_tests/test: 1026 passed, 0 skipped. The 9 earlier skips were the Docker check tests, which now run.
• Mutations: an unconditional skip in the Docker read (fails the read tests in both test_docker_read.py and test_scenario_sections.py), dropping the unknown-config or unknown-stream check, dropping the basic_read addition, and dropping the working-directory arguments from check, discover or read each fail tests.
• ruff format, ruff check and mypy clean on the changed modules
• Earlier revision: Docker suite on 11 published connector images and the 58 Python sources with an acceptance-test-config.yml. Not re-run after the default flip, since that run used explicit option values.
• Test Connectors matrix on this PR green, with the Docker read on in image test

🤖 Generated with Claude Code

[github-actions]

👋 Greetings, Airbyte Team Member!

Here are some helpful tips and reminders for your convenience.

💡 Show Tips and Tricks

Testing This CDK Version

You can test this version of the CDK using the following:

# Run the CLI from this branch:
uvx 'git+https://github.com/airbytehq/airbyte-python-cdk.git@tolik0/cdk/enable-docker-read-test#egg=airbyte-python-cdk[dev]' --help

# Update a connector to use the CDK from this branch ref:
cd airbyte-integrations/connectors/source-example
poe use-cdk-branch tolik0/cdk/enable-docker-read-test

PR Slash Commands

Airbyte Maintainers can execute the following slash commands on your PR:

• /autofix - Fixes most formatting and linting issues
• /poetry-lock - Updates poetry.lock file
• /test - Runs connector tests with the updated CDK
• /prerelease - Triggers a prerelease publish with default arguments
• /poe build - Regenerate git-committed build artifacts, such as the pydantic models which are generated from the manifest JSON schema in YAML.
• /poe <command> - Runs any poe command in the CDK environment

📚 Show Repo Guidance

Helpful Resources

• CDK API Reference

📝 Edit this welcome message.

[github-actions]

PyTest Results (Fast)

6 223 tests  +39   6 201 ✅ +48   6m 34s ⏱️ +35s
    1 suites ± 0      22 💤  -  9 
    1 files   ± 0       0 ❌ ± 0 

Results for commit e8112d7. ± Comparison against base commit 18fc1de.

♻️ This comment has been updated with latest results.

[github-actions]

PyTest Results (Full)

6 226 tests  +39   6 204 ✅ +48   9m 43s ⏱️ - 2m 2s
    1 suites ± 0      22 💤  -  9 
    1 files   ± 0       0 ❌ ± 0 

Results for commit e8112d7. ± Comparison against base commit 18fc1de.

♻️ This comment has been updated with latest results.

[Daryna Ishchenko (darynaishchenko)]

Approve. I didn't find a breaking change. The Docker read test was already off by default (--read-from-streams was registered with default=None, which the fixture mapped to none) and still is. When it's on, it now makes the same assert_read_records and stream-status checks as test_basic_read. The only change every connector gets is the /source working directory for Docker check and discover, and that matches the platform (ReplicationContainerFactory uses withWorkingDir(SOURCE_DIR), ConnectorPodFactory uses withWorkingDir(configDir)).

Verified:

• The 216 tests in the touched test files pass locally.
• I broke the code on purpose 35 ways, and 31 of those fail a test.
• The matrix ran the Docker read with the tmpfs args, and it passed on shopify, intercom, pokeapi and hardcoded-records.
• destination-motherduck is red at the secret fetch, as on most recent PRs.

The two 🟠 comments below aren't blocking. Rollout: #88939's fleet dry run on the old-CDK images is the real gate, because those images now face the 7.42.0 state and primary-key checks.