Skip to content

Increase envtest timeouts and resource limits for CI#1798

Merged
openshift-merge-bot[bot] merged 1 commit intoopenstack-k8s-operators:mainfrom
stuggi:dmn_test
Feb 6, 2026
Merged

Increase envtest timeouts and resource limits for CI#1798
openshift-merge-bot[bot] merged 1 commit intoopenstack-k8s-operators:mainfrom
stuggi:dmn_test

Conversation

@stuggi
Copy link
Contributor

@stuggi stuggi commented Feb 6, 2026

Functional tests were failing during CRD installation with etcd
timeouts in CI environments. The error "etcdserver: request timed out"
indicates etcd couldn't process CRD creation requests within its
default 5-second timeout, likely due to resource constraints (slow
disk I/O, CPU pressure, or memory limits in CI).

Configuration changes to envtest.Environment:

  1. Client-side timeouts:

    • ControlPlaneStartTimeout: 2 minutes
    • ControlPlaneStopTimeout: 2 minutes
    • CRDInstallOptions.MaxTime: 5 minutes
  2. API Server configuration:

    • request-timeout: 5 minutes (allow longer-running requests)
    • max-requests-inflight: 800 (increased from default 400)
    • max-mutating-requests-inflight: 400 (increased from default 200)
  3. Etcd configuration:

    • quota-backend-bytes: 8GB (increased from default 2GB)

These changes accommodate the large number of CRDs being installed
(100+ CRDs from cinder, glance, nova, neutron, and other operators)
and resource-constrained CI environments.

Errors observed:

  • etcdserver: request timed out
  • unable to create CRD "openstackdataplanenodesets.dataplane.openstack.org"
  • unable to create CRD "routes.route.openshift.io"

@openshift-ci openshift-ci bot requested review from dprince and rabi February 6, 2026 12:48
@openshift-ci openshift-ci bot added the approved label Feb 6, 2026
@stuggi stuggi force-pushed the dmn_test branch 3 times, most recently from 47dc015 to bc1b10a Compare February 6, 2026 13:37
Functional tests were failing during CRD installation with etcd
timeouts in CI environments. The error "etcdserver: request timed out"
indicates etcd couldn't process CRD creation requests within its
default 5-second timeout, likely due to resource constraints (slow
disk I/O, CPU pressure, or memory limits in CI).

Configuration changes to envtest.Environment:

1. Client-side timeouts:
   - ControlPlaneStartTimeout: 2 minutes
   - ControlPlaneStopTimeout: 2 minutes
   - CRDInstallOptions.MaxTime: 5 minutes

2. API Server configuration:
   - request-timeout: 5 minutes (allow longer-running requests)
   - max-requests-inflight: 800 (increased from default 400)
   - max-mutating-requests-inflight: 400 (increased from default 200)

3. Etcd configuration:
   - quota-backend-bytes: 8GB (increased from default 2GB)

These changes accommodate the large number of CRDs being installed
(100+ CRDs from cinder, glance, nova, neutron, and other operators)
and resource-constrained CI environments.

Errors observed:
  - etcdserver: request timed out
  - unable to create CRD "openstackdataplanenodesets.dataplane.openstack.org"
  - unable to create CRD "routes.route.openshift.io"
@stuggi stuggi changed the title [DNM] test Increase envtest timeouts and resource limits for CI Feb 6, 2026
@lmiccini
Copy link
Contributor

lmiccini commented Feb 6, 2026

/test functional

@stuggi
Copy link
Contributor Author

stuggi commented Feb 6, 2026

/override ci/prow/openstack-operator-build-deploy-kuttl-4-18

@openshift-ci
Copy link
Contributor

openshift-ci bot commented Feb 6, 2026

@stuggi: Overrode contexts on behalf of stuggi: ci/prow/openstack-operator-build-deploy-kuttl-4-18

Details

In response to this:

/override ci/prow/openstack-operator-build-deploy-kuttl-4-18

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@lmiccini
Copy link
Contributor

lmiccini commented Feb 6, 2026

/lgtm

@openshift-ci
Copy link
Contributor

openshift-ci bot commented Feb 6, 2026

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: dprince, stuggi

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-merge-bot openshift-merge-bot bot merged commit adad2c1 into openstack-k8s-operators:main Feb 6, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants