Skip to content

[FLINK-40662][tests] Fail fast when the S3 test container does not start - #29222

Open
MartijnVisser wants to merge 2 commits into
apache:masterfrom
MartijnVisser:FLINK-40662-s3-container-start
Open

MartijnVisser wants to merge 2 commits into
apache:masterfrom
MartijnVisser:FLINK-40662-s3-container-start

Conversation

@MartijnVisser

Copy link
Copy Markdown
Contributor

What is the purpose of the change

  • s3_start missed the docker run exit code and never waited for the container, so a failed image
    pull left S3_ENDPOINT as the bare string http:// and 'Run kubernetes Materialized Table test'
    failed with a null session handle.

Brief change log

  • s3_start keeps the docker run exit code, retries three times, aborts if the container does not
    come up, and rejects an endpoint without a port binding
  • New wait_for_container_running in common.sh, used by s3_start and aws_cli_start
  • test_kubernetes_materialized_table.sh rejects an empty or null session handle

Verifying this change

This change added tests and can be verified as follows:

  • s3_start against a docker stub reproducing the failed pulls of azure 79037 and 79076: 12 of 12
    red on 2440f65, 4 of 4 green after
  • against real docker: the happy path serves /healthz, a missing image tag aborts
  • the e2e test itself needs minikube with --vm-driver=none and does not run on macOS

Does this pull request potentially affect one of the following parts:

  • Dependencies (does it add or upgrade a dependency): no
  • The public API, i.e., is any changed class annotated with @Public(Evolving): no
  • The serializers: no
  • The runtime per-record code paths (performance sensitive): no
  • Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
  • The S3 file system connector: no

Documentation

  • Does this pull request introduce a new feature? no
  • If yes, how is the feature documented? not applicable

Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)

Generated-by: Claude Code (Claude Opus 5)

`export VAR=$(docker run ...)` reports the exit code of export, and the readiness
loop compared with `-ne`, which is arithmetic, so a failed image pull left
S3_ENDPOINT as the bare string `http://`.

* Keep the `docker run` exit code and retry the start three times
* Add `wait_for_container_running` to `common.sh`, used here and by `aws_cli_start`
* Reject an endpoint without a port binding and bound the readiness wait

Generated-by: Claude Code (Claude Opus 5)
A failed openSession yielded the string "null" and the test carried on, sending
every request to /sessions/null/... The handle is now checked the way
execute_statement already checks the operation handle.

Generated-by: Claude Code (Claude Opus 5)
@flinkbot

flinkbot commented Sep 17, 2026

Copy link
Copy Markdown
Collaborator

CI report:

Bot commands The @flinkbot bot supports the following commands:
  • @flinkbot run azure re-run the last Azure build

if [[ "$(docker inspect -f '{{.State.Running}}' "${container_id}" 2>/dev/null)" == "true" ]]; then
return 0
fi
sleep 0.1

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

optional qq: though not a issue in happy scenario, isnt 0.1s a little too less of sleep? Maybe 1s.

@github-actions github-actions Bot added the community-reviewed PR has been reviewed by the community. label Sep 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

community-reviewed PR has been reviewed by the community.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants