CHORE: bump controller-gen/envtest and set --use-deprecated-gcs in Makefile recipe to false by kaizakin · Pull Request #281 · shipwright-io/operator

kaizakin · 2026-02-26T20:29:07Z

Changes

bumped the controller-gen/envtest from v0.15.0 to v0.18.0
added a flag --use-deprecated-gcs=false tells the discovery of the binaries to not use the legacy API's
bumped the setup-envtest from release-0.18 to release-0.22

Fixes #280

There are some autogenerated changes with this PR which im not sure about. :)

/kind cleanup

Submitter Checklist

Includes tests if functionality changed/was added
Includes docs if changes are user-facing
Set a kind label on this PR
Release notes block has been filled in, or marked NONE

See the contributor guide
for details on coding conventions, github and prow interactions, and the code review process.

Release Notes

NONE

kaizakin · 2026-02-26T20:31:19Z

Requesting review: @IrvingMg @hasanawad94

IrvingMg

Thank you for sending the fix. However, I’m not sure I fully understood the issue. I see that you encountered an error while running make test locally as well, but was it the same issue that appeared in CI?

Also, what about the unauthorized error? Was it related to this issue in any way?

It would be great to better understand the root cause, in case there’s something we can do to prevent this from happening again in the future or in the other repositories.

kaizakin · 2026-02-27T15:21:49Z

Yep, @IrvingMg i saw the same unauthorized issue while i was running make test locally

Actually there were two errors when i ran make test locally, one is this

../../../go/pkg/mod/golang.org/x/tools@v0.20.0/internal/tokeninternal/tokeninternal.go:64:9: invalid array length -delta * delta (constant -256 of type int64)
make: *** [Makefile:208: controller-gen] Error 1

which was solved by bumping the controller-gen version

And the second one is this,

In my understanding setupenvtest is trying to pull binaries from deprecated endpoints
since no binaries could be downloaded envtest puts nothing in the path
So PATH variable is unset which in turn causes the control plane creation failure as we saw in the CI logs

using --use-deprecated-gcs=false solved the issue locally

I made a ad hoc fix by searching here and there. surprisingly it worked
let's hear from more community members and make sure this doesn't happen again.

IrvingMg · 2026-02-27T15:48:07Z

It’s interesting, I don’t see the following error in the CI logs:

../../../go/pkg/mod/golang.org/x/tools@v0.20.0/internal/tokeninternal/tokeninternal.go:64:9: invalid array length -delta * delta (constant -256 of type int64)
make: *** [Makefile:208: controller-gen] Error 1

Specifically, I don’t see it in the CI run here:
https://github.com/shipwright-io/operator/actions/runs/22130376447/job/64657021190?pr=278

I assume there may be some environment differences. I’m trying to understand the difference so we can keep CI and local environments aligned. Do you have any idea what might be different? For example, could the CI images be outdated?

Also, could you share the full logs from your local make test run?

kaizakin · 2026-02-27T17:10:49Z

Thank you so much for keeping up and helping me with this issue @IrvingMg

Here are the local make test logs

# golang.org/x/tools/internal/tokeninternal
../../../go/pkg/mod/golang.org/x/tools@v0.20.0/internal/tokeninternal/tokeninternal.go:64:9: invalid array length -delta * delta (constant -256 of type int64)
make: *** [Makefile:208: controller-gen] Error 1

kaizakin · 2026-02-27T17:19:07Z

Surprisingly after bumping the controller-gen version I'm not seeing the same unauthorized error I saw yesterday

Now tests seem to work perfectly fine locally

•••••••

Ran 7 of 7 Specs in 16.682 seconds
SUCCESS! -- 7 Passed | 0 Failed | 0 Pending | 0 Skipped

For context This was the error i was facing yesterday

/home/karthik/Projects/oss/operator/bin/controller-gen rbac:roleName=manager-role crd webhook paths="./..." output:crd:artifacts:config=config/crd/bases
/home/karthik/Projects/oss/operator/bin/controller-gen object:headerFile="hack/boilerplate.go.txt" paths="./..."
go fmt ./...
go vet ./...
go: downloading sigs.k8s.io/controller-runtime v0.18.7
go: downloading sigs.k8s.io/controller-runtime/tools/setup-envtest v0.0.0-20250106171007-42f791873869
go: downloading go.uber.org/zap v1.26.0
go: downloading sigs.k8s.io/yaml v1.3.0
go: downloading go.uber.org/multierr v1.10.0
unable to list versions to find latest one: unable list bucket items -- got status "401 Unauthorized" from GCS
KO_DATA_PATH=/home/karthik/Projects/oss/operator/kodata KUBEBUILDER_ASSETS="" SKIP_ENVTEST=false go test ./... -coverprofile cover.out -failfast -test.v -test.failfast
	github.com/shipwright-io/operator		coverage: 0.0% of statements
=== RUN   TestIsReady
--- PASS: TestIsReady (0.00s)
PASS
coverage: 9.8% of statements
ok  	github.com/shipwright-io/operator/api/v1alpha1	0.004s	coverage: 9.8% of statements
=== RUN   TestShipwrightBuildReconciler_Finalizers
=== RUN   TestShipwrightBuildReconciler_Finalizers/setupManifestival
{"level":"info","ts":"2026-02-27T00:44:31+05:30","msg":"Parsing manifest"}
{"level":"info","ts":"2026-02-27T00:44:31+05:30","msg":"Parsing manifest"}
=== RUN   TestShipwrightBuildReconciler_Finalizers/setFinalizer
=== RUN   TestShipwrightBuildReconciler_Finalizers/unsetFinalizer
--- PASS: TestShipwrightBuildReconciler_Finalizers (0.16s)
    --- PASS: TestShipwrightBuildReconciler_Finalizers/setupManifestival (0.16s)
    --- PASS: TestShipwrightBuildReconciler_Finalizers/setFinalizer (0.00s)
    --- PASS: TestShipwrightBuildReconciler_Finalizers/unsetFinalizer (0.00s)
=== RUN   TestShipwrightBuildReconciler_Reconcile
=== RUN   TestShipwrightBuildReconciler_Reconcile/target_namespace_is_informed
    shipwrightbuild_controller_test.go:82: Creating test namespace 'namespace'
=== RUN   TestShipwrightBuildReconciler_Reconcile/target_namespace_is_informed/create-test-namespace
=== RUN   TestShipwrightBuildReconciler_Reconcile/target_namespace_is_informed/setupManifestival
{"level":"info","ts":"2026-02-27T00:44:31+05:30","msg":"Parsing manifest"}
{"level":"info","ts":"2026-02-27T00:44:31+05:30","msg":"Parsing manifest"}
=== NAME  TestShipwrightBuildReconciler_Reconcile/target_namespace_is_informed
    shipwrightbuild_controller_test.go:166: Deploying Shipwright Controller against 'namespace' namespace
=== RUN   TestShipwrightBuildReconciler_Reconcile/target_namespace_is_informed/rollout-manifests
    shipwrightbuild_controller_test.go:171: k8s 1.27 upgrade led to test clients returning false not found errors
=== RUN   TestShipwrightBuildReconciler_Reconcile/target_namespace_is_informed/rollout-manifests-with-images-env-vars
    shipwrightbuild_controller_test.go:190: k8s 1.27 upgrade led to test clients returning false not found errors
=== RUN   TestShipwrightBuildReconciler_Reconcile/target_namespace_is_informed/rollback-manifests
    shipwrightbuild_controller_test.go:216: k8s 1.27 upgrade led to test clients returning false not found errors
=== RUN   TestShipwrightBuildReconciler_Reconcile/target_namespace_is_not_informed
    shipwrightbuild_controller_test.go:82: Creating test namespace 'shipwright-build'
=== RUN   TestShipwrightBuildReconciler_Reconcile/target_namespace_is_not_informed/create-test-namespace
=== RUN   TestShipwrightBuildReconciler_Reconcile/target_namespace_is_not_informed/setupManifestival
{"level":"info","ts":"2026-02-27T00:44:31+05:30","msg":"Parsing manifest"}
{"level":"info","ts":"2026-02-27T00:44:31+05:30","msg":"Parsing manifest"}
=== NAME  TestShipwrightBuildReconciler_Reconcile/target_namespace_is_not_informed
    shipwrightbuild_controller_test.go:166: Deploying Shipwright Controller against 'shipwright-build' namespace
=== RUN   TestShipwrightBuildReconciler_Reconcile/target_namespace_is_not_informed/rollout-manifests
    shipwrightbuild_controller_test.go:171: k8s 1.27 upgrade led to test clients returning false not found errors
=== RUN   TestShipwrightBuildReconciler_Reconcile/target_namespace_is_not_informed/rollout-manifests-with-images-env-vars
    shipwrightbuild_controller_test.go:190: k8s 1.27 upgrade led to test clients returning false not found errors
=== RUN   TestShipwrightBuildReconciler_Reconcile/target_namespace_is_not_informed/rollback-manifests
    shipwrightbuild_controller_test.go:216: k8s 1.27 upgrade led to test clients returning false not found errors
--- PASS: TestShipwrightBuildReconciler_Reconcile (0.26s)
    --- PASS: TestShipwrightBuildReconciler_Reconcile/target_namespace_is_informed (0.14s)
        --- PASS: TestShipwrightBuildReconciler_Reconcile/target_namespace_is_informed/create-test-namespace (0.00s)
        --- PASS: TestShipwrightBuildReconciler_Reconcile/target_namespace_is_informed/setupManifestival (0.14s)
        --- SKIP: TestShipwrightBuildReconciler_Reconcile/target_namespace_is_informed/rollout-manifests (0.00s)
        --- SKIP: TestShipwrightBuildReconciler_Reconcile/target_namespace_is_informed/rollout-manifests-with-images-env-vars (0.00s)
        --- SKIP: TestShipwrightBuildReconciler_Reconcile/target_namespace_is_informed/rollback-manifests (0.00s)
    --- PASS: TestShipwrightBuildReconciler_Reconcile/target_namespace_is_not_informed (0.12s)
        --- PASS: TestShipwrightBuildReconciler_Reconcile/target_namespace_is_not_informed/create-test-namespace (0.00s)
        --- PASS: TestShipwrightBuildReconciler_Reconcile/target_namespace_is_not_informed/setupManifestival (0.12s)
        --- SKIP: TestShipwrightBuildReconciler_Reconcile/target_namespace_is_not_informed/rollout-manifests (0.00s)
        --- SKIP: TestShipwrightBuildReconciler_Reconcile/target_namespace_is_not_informed/rollout-manifests-with-images-env-vars (0.00s)
        --- SKIP: TestShipwrightBuildReconciler_Reconcile/target_namespace_is_not_informed/rollback-manifests (0.00s)
=== RUN   TestShipwrightBuildReconciler_OperandReadiness
    shipwrightbuild_controller_test.go:82: Creating test namespace 'namespace'
=== RUN   TestShipwrightBuildReconciler_OperandReadiness/create-test-namespace
=== RUN   TestShipwrightBuildReconciler_OperandReadiness/setupManifestival
{"level":"info","ts":"2026-02-27T00:44:31+05:30","msg":"Parsing manifest"}
{"level":"info","ts":"2026-02-27T00:44:31+05:30","msg":"Parsing manifest"}
{"level":"info","ts":"2026-02-27T00:44:31+05:30","msg":"Starting resource reconciliation...","namespace":"default","name":"name"}
{"level":"info","ts":"2026-02-27T00:44:31+05:30","msg":"TektonConfig is not ready yet","namespace":"default","name":"name"}
{"level":"info","ts":"2026-02-27T00:44:31+05:30","msg":"TektonConfig is not ready, requeueing request","namespace":"default","name":"name"}
{"level":"info","ts":"2026-02-27T00:44:31+05:30","msg":"Starting resource reconciliation...","namespace":"default","name":"name"}
{"level":"info","ts":"2026-02-27T00:44:31+05:30","msg":"TektonConfig is ready","namespace":"default","name":"name"}
{"level":"info","ts":"2026-02-27T00:44:32+05:30","msg":"Applying manifest's resources...","namespace":"default","name":"name","targetNamespace":"namespace"}
{"level":"info","ts":"2026-02-27T00:44:32+05:30","msg":"Creating","name":"namespace/shipwright-build-controller","type":"rbac.authorization.k8s.io/v1, Kind=Role"}
{"level":"info","ts":"2026-02-27T00:44:32+05:30","msg":"Creating","name":"/shipwright-build-controller","type":"rbac.authorization.k8s.io/v1, Kind=ClusterRole"}
{"level":"info","ts":"2026-02-27T00:44:32+05:30","msg":"Creating","name":"/shipwright-build-controller","type":"rbac.authorization.k8s.io/v1, Kind=ClusterRoleBinding"}
{"level":"info","ts":"2026-02-27T00:44:32+05:30","msg":"Creating","name":"namespace/shipwright-build-controller","type":"rbac.authorization.k8s.io/v1, Kind=RoleBinding"}
{"level":"info","ts":"2026-02-27T00:44:32+05:30","msg":"Creating","name":"namespace/shipwright-build-controller","type":"/v1, Kind=ServiceAccount"}
{"level":"info","ts":"2026-02-27T00:44:32+05:30","msg":"Creating","name":"namespace/shipwright-build-webhook","type":"/v1, Kind=ServiceAccount"}
{"level":"info","ts":"2026-02-27T00:44:32+05:30","msg":"Creating","name":"namespace/shp-build-webhook","type":"/v1, Kind=Service"}
{"level":"info","ts":"2026-02-27T00:44:32+05:30","msg":"Creating","name":"namespace/shipwright-build-controller","type":"apps/v1, Kind=Deployment"}
{"level":"info","ts":"2026-02-27T00:44:32+05:30","msg":"Creating","name":"namespace/shipwright-build-webhook","type":"apps/v1, Kind=Deployment"}
{"level":"info","ts":"2026-02-27T00:44:32+05:30","msg":"Creating","name":"/shipwright-build-aggregate-edit","type":"rbac.authorization.k8s.io/v1, Kind=ClusterRole"}
{"level":"info","ts":"2026-02-27T00:44:32+05:30","msg":"Creating","name":"/shipwright-build-aggregate-view","type":"rbac.authorization.k8s.io/v1, Kind=ClusterRole"}
{"level":"info","ts":"2026-02-27T00:44:32+05:30","msg":"Creating","name":"/buildruns.shipwright.io","type":"apiextensions.k8s.io/v1, Kind=CustomResourceDefinition"}
{"level":"info","ts":"2026-02-27T00:44:32+05:30","msg":"Creating","name":"/builds.shipwright.io","type":"apiextensions.k8s.io/v1, Kind=CustomResourceDefinition"}
{"level":"info","ts":"2026-02-27T00:44:32+05:30","msg":"Creating","name":"/buildstrategies.shipwright.io","type":"apiextensions.k8s.io/v1, Kind=CustomResourceDefinition"}
{"level":"info","ts":"2026-02-27T00:44:32+05:30","msg":"Creating","name":"/clusterbuildstrategies.shipwright.io","type":"apiextensions.k8s.io/v1, Kind=CustomResourceDefinition"}
{"level":"info","ts":"2026-02-27T00:44:32+05:30","msg":"Creating","name":"/buildah-shipwright-managed-push","type":"shipwright.io/v1beta1, Kind=ClusterBuildStrategy"}
{"level":"info","ts":"2026-02-27T00:44:32+05:30","msg":"Creating","name":"/buildah-strategy-managed-push","type":"shipwright.io/v1beta1, Kind=ClusterBuildStrategy"}
{"level":"info","ts":"2026-02-27T00:44:32+05:30","msg":"Creating","name":"/buildkit","type":"shipwright.io/v1beta1, Kind=ClusterBuildStrategy"}
{"level":"info","ts":"2026-02-27T00:44:32+05:30","msg":"Creating","name":"/buildpacks-v3-heroku","type":"shipwright.io/v1beta1, Kind=ClusterBuildStrategy"}
{"level":"info","ts":"2026-02-27T00:44:32+05:30","msg":"Creating","name":"/buildpacks-v3","type":"shipwright.io/v1beta1, Kind=ClusterBuildStrategy"}
{"level":"info","ts":"2026-02-27T00:44:32+05:30","msg":"Creating","name":"/kaniko","type":"shipwright.io/v1beta1, Kind=ClusterBuildStrategy"}
{"level":"info","ts":"2026-02-27T00:44:32+05:30","msg":"Creating","name":"/ko","type":"shipwright.io/v1beta1, Kind=ClusterBuildStrategy"}
{"level":"info","ts":"2026-02-27T00:44:32+05:30","msg":"Creating","name":"/source-to-image-redhat","type":"shipwright.io/v1beta1, Kind=ClusterBuildStrategy"}
{"level":"info","ts":"2026-02-27T00:44:32+05:30","msg":"Creating","name":"/source-to-image","type":"shipwright.io/v1beta1, Kind=ClusterBuildStrategy"}
{"level":"info","ts":"2026-02-27T00:44:32+05:30","msg":"All done!","namespace":"default","name":"name","targetNamespace":"namespace"}
--- PASS: TestShipwrightBuildReconciler_OperandReadiness (0.24s)
    --- PASS: TestShipwrightBuildReconciler_OperandReadiness/create-test-namespace (0.00s)
    --- PASS: TestShipwrightBuildReconciler_OperandReadiness/setupManifestival (0.19s)
=== RUN   TestAPIs
Running Suite: Controller Suite - /home/karthik/Projects/oss/operator/controllers
=================================================================================
Random Seed: 1772133271

Will run 7 of 7 specs
------------------------------
[BeforeSuite] [FAILED] [0.011 seconds]
[BeforeSuite] 
/home/karthik/Projects/oss/operator/controllers/suite_test.go:243

  Timeline >>
  STEP: bootstrapping test environment @ 02/27/26 00:44:32.04
  2026-02-27T00:44:32+05:30	DEBUG	controller-runtime.test-env	starting control plane
  2026-02-27T00:44:32+05:30	ERROR	controller-runtime.test-env	unable to start the controlplane	{"tries": 0, "error": "exec: \"etcd\": executable file not found in $PATH"}
  sigs.k8s.io/controller-runtime/pkg/envtest.(*Environment).startControlPlane
  	/home/karthik/Projects/oss/operator/vendor/sigs.k8s.io/controller-runtime/pkg/envtest/server.go:366
  sigs.k8s.io/controller-runtime/pkg/envtest.(*Environment).Start
  	/home/karthik/Projects/oss/operator/vendor/sigs.k8s.io/controller-runtime/pkg/envtest/server.go:279
  github.com/shipwright-io/operator/controllers.init.func3
  	/home/karthik/Projects/oss/operator/controllers/suite_test.go:252
  github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func3
  	/home/karthik/Projects/oss/operator/vendor/github.com/onsi/ginkgo/v2/internal/node.go:524
  github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func3
  	/home/karthik/Projects/oss/operator/vendor/github.com/onsi/ginkgo/v2/internal/suite.go:942
  2026-02-27T00:44:32+05:30	ERROR	controller-runtime.test-env	unable to start the controlplane	{"tries": 1, "error": "exec: \"etcd\": executable file not found in $PATH"}
  sigs.k8s.io/controller-runtime/pkg/envtest.(*Environment).startControlPlane
  	/home/karthik/Projects/oss/operator/vendor/sigs.k8s.io/controller-runtime/pkg/envtest/server.go:366
  sigs.k8s.io/controller-runtime/pkg/envtest.(*Environment).Start
  	/home/karthik/Projects/oss/operator/vendor/sigs.k8s.io/controller-runtime/pkg/envtest/server.go:279
  github.com/shipwright-io/operator/controllers.init.func3
  	/home/karthik/Projects/oss/operator/controllers/suite_test.go:252
  github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func3
  	/home/karthik/Projects/oss/operator/vendor/github.com/onsi/ginkgo/v2/internal/node.go:524
  github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func3
  	/home/karthik/Projects/oss/operator/vendor/github.com/onsi/ginkgo/v2/internal/suite.go:942
  2026-02-27T00:44:32+05:30	ERROR	controller-runtime.test-env	unable to start the controlplane	{"tries": 2, "error": "exec: \"etcd\": executable file not found in $PATH"}
  sigs.k8s.io/controller-runtime/pkg/envtest.(*Environment).startControlPlane
  	/home/karthik/Projects/oss/operator/vendor/sigs.k8s.io/controller-runtime/pkg/envtest/server.go:366
  sigs.k8s.io/controller-runtime/pkg/envtest.(*Environment).Start
  	/home/karthik/Projects/oss/operator/vendor/sigs.k8s.io/controller-runtime/pkg/envtest/server.go:279
  github.com/shipwright-io/operator/controllers.init.func3
  	/home/karthik/Projects/oss/operator/controllers/suite_test.go:252
  github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func3
  	/home/karthik/Projects/oss/operator/vendor/github.com/onsi/ginkgo/v2/internal/node.go:524
  github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func3
  	/home/karthik/Projects/oss/operator/vendor/github.com/onsi/ginkgo/v2/internal/suite.go:942
  2026-02-27T00:44:32+05:30	ERROR	controller-runtime.test-env	unable to start the controlplane	{"tries": 3, "error": "exec: \"etcd\": executable file not found in $PATH"}
  sigs.k8s.io/controller-runtime/pkg/envtest.(*Environment).startControlPlane
  	/home/karthik/Projects/oss/operator/vendor/sigs.k8s.io/controller-runtime/pkg/envtest/server.go:366
  sigs.k8s.io/controller-runtime/pkg/envtest.(*Environment).Start
  	/home/karthik/Projects/oss/operator/vendor/sigs.k8s.io/controller-runtime/pkg/envtest/server.go:279
  github.com/shipwright-io/operator/controllers.init.func3
  	/home/karthik/Projects/oss/operator/controllers/suite_test.go:252
  github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func3
  	/home/karthik/Projects/oss/operator/vendor/github.com/onsi/ginkgo/v2/internal/node.go:524
  github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func3
  	/home/karthik/Projects/oss/operator/vendor/github.com/onsi/ginkgo/v2/internal/suite.go:942
  2026-02-27T00:44:32+05:30	ERROR	controller-runtime.test-env	unable to start the controlplane	{"tries": 4, "error": "exec: \"etcd\": executable file not found in $PATH"}
  sigs.k8s.io/controller-runtime/pkg/envtest.(*Environment).startControlPlane
  	/home/karthik/Projects/oss/operator/vendor/sigs.k8s.io/controller-runtime/pkg/envtest/server.go:366
  sigs.k8s.io/controller-runtime/pkg/envtest.(*Environment).Start
  	/home/karthik/Projects/oss/operator/vendor/sigs.k8s.io/controller-runtime/pkg/envtest/server.go:279
  github.com/shipwright-io/operator/controllers.init.func3
  	/home/karthik/Projects/oss/operator/controllers/suite_test.go:252
  github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func3
  	/home/karthik/Projects/oss/operator/vendor/github.com/onsi/ginkgo/v2/internal/node.go:524
  github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func3
  	/home/karthik/Projects/oss/operator/vendor/github.com/onsi/ginkgo/v2/internal/suite.go:942
  [FAILED] in [BeforeSuite] - /home/karthik/Projects/oss/operator/controllers/suite_test.go:253 @ 02/27/26 00:44:32.051
  << Timeline

  [FAILED] Unexpected error:
      <*fmt.wrapError | 0xc0015028a0>: 
      unable to start control plane itself: failed to start the controlplane. retried 5 times: exec: "etcd": executable file not found in $PATH
      {
          msg: "unable to start control plane itself: failed to start the controlplane. retried 5 times: exec: \"etcd\": executable file not found in $PATH",
          err: <*fmt.wrapError | 0xc001502880>{
              msg: "failed to start the controlplane. retried 5 times: exec: \"etcd\": executable file not found in $PATH",
              err: <*exec.Error | 0xc001502840>{
                  Name: "etcd",
                  Err: <*errors.errorString | 0x4076890>{
                      s: "executable file not found in $PATH",
                  },
              },
          },
      }
  occurred
  In [BeforeSuite] at: /home/karthik/Projects/oss/operator/controllers/suite_test.go:253 @ 02/27/26 00:44:32.051
------------------------------
[AfterSuite] [PANICKED] [0.000 seconds]
[AfterSuite] 
/home/karthik/Projects/oss/operator/controllers/suite_test.go:299

  [PANICKED] Test Panicked
  In [AfterSuite] at: /usr/local/go/src/runtime/panic.go:262 @ 02/27/26 00:44:32.051

  runtime error: invalid memory address or nil pointer dereference

  Full Stack Trace
    github.com/shipwright-io/operator/controllers.init.func4()
    	/home/karthik/Projects/oss/operator/controllers/suite_test.go:300 +0x15
------------------------------

Summarizing 2 Failures:
  [FAIL] [BeforeSuite] 
  /home/karthik/Projects/oss/operator/controllers/suite_test.go:253
  [PANICKED!] [AfterSuite] 
  /usr/local/go/src/runtime/panic.go:262

Ran 0 of 7 Specs in 0.012 seconds
FAIL! -- A BeforeSuite node failed so all tests were skipped.
--- FAIL: TestAPIs (0.01s)
FAIL
coverage: 42.8% of statements
FAIL	github.com/shipwright-io/operator/controllers	0.768s
=== RUN   TestReconcileBuildStrategies
=== RUN   TestReconcileBuildStrategies/no_Shipwright_CRDs
{"level":"info","ts":"2026-02-27T00:44:31+05:30","msg":"Parsing manifest"}
=== RUN   TestReconcileBuildStrategies/install_Shipwright_CRDs
{"level":"info","ts":"2026-02-27T00:44:31+05:30","msg":"Parsing manifest"}
{"level":"info","ts":"2026-02-27T00:44:31+05:30","msg":"Creating","name":"/buildah-shipwright-managed-push","type":"shipwright.io/v1beta1, Kind=ClusterBuildStrategy"}
{"level":"info","ts":"2026-02-27T00:44:31+05:30","msg":"Creating","name":"/buildah-strategy-managed-push","type":"shipwright.io/v1beta1, Kind=ClusterBuildStrategy"}
{"level":"info","ts":"2026-02-27T00:44:31+05:30","msg":"Creating","name":"/buildkit","type":"shipwright.io/v1beta1, Kind=ClusterBuildStrategy"}
{"level":"info","ts":"2026-02-27T00:44:31+05:30","msg":"Creating","name":"/buildpacks-v3-heroku","type":"shipwright.io/v1beta1, Kind=ClusterBuildStrategy"}
{"level":"info","ts":"2026-02-27T00:44:31+05:30","msg":"Creating","name":"/buildpacks-v3","type":"shipwright.io/v1beta1, Kind=ClusterBuildStrategy"}
{"level":"info","ts":"2026-02-27T00:44:31+05:30","msg":"Creating","name":"/kaniko","type":"shipwright.io/v1beta1, Kind=ClusterBuildStrategy"}
{"level":"info","ts":"2026-02-27T00:44:31+05:30","msg":"Creating","name":"/ko","type":"shipwright.io/v1beta1, Kind=ClusterBuildStrategy"}
{"level":"info","ts":"2026-02-27T00:44:31+05:30","msg":"Creating","name":"/source-to-image-redhat","type":"shipwright.io/v1beta1, Kind=ClusterBuildStrategy"}
{"level":"info","ts":"2026-02-27T00:44:31+05:30","msg":"Creating","name":"/source-to-image","type":"shipwright.io/v1beta1, Kind=ClusterBuildStrategy"}
    buildstrategy_test.go:78: build strategies: [buildah-shipwright-managed-push buildah-strategy-managed-push buildkit buildpacks-v3-heroku buildpacks-v3 kaniko ko source-to-image-redhat source-to-image]
--- PASS: TestReconcileBuildStrategies (0.08s)
    --- PASS: TestReconcileBuildStrategies/no_Shipwright_CRDs (0.02s)
    --- PASS: TestReconcileBuildStrategies/install_Shipwright_CRDs (0.05s)
PASS
coverage: 77.8% of statements
ok  	github.com/shipwright-io/operator/pkg/buildstrategy	0.124s	coverage: 77.8% of statements
=== RUN   TestReconcileCertManager
=== RUN   TestReconcileCertManager/No_cert-manager_Objects
=== RUN   TestReconcileCertManager/cert-manager_certificates_defined
{"level":"info","ts":"2026-02-27T00:44:31+05:30","msg":"Parsing manifest"}
{"level":"info","ts":"2026-02-27T00:44:31+05:30","msg":"Creating","name":"shipwright-build/selfsigned-issuer","type":"cert-manager.io/v1, Kind=Issuer"}
{"level":"info","ts":"2026-02-27T00:44:31+05:30","msg":"Creating","name":"shipwright-build/shipwright-build-webhook-cert","type":"cert-manager.io/v1, Kind=Certificate"}
=== RUN   TestReconcileCertManager/cert-manager_crd_defined
{"level":"info","ts":"2026-02-27T00:44:31+05:30","msg":"Parsing manifest"}
{"level":"info","ts":"2026-02-27T00:44:31+05:30","msg":"Creating","name":"shipwright-build/selfsigned-issuer","type":"cert-manager.io/v1, Kind=Issuer"}
{"level":"info","ts":"2026-02-27T00:44:31+05:30","msg":"Creating","name":"shipwright-build/shipwright-build-webhook-cert","type":"cert-manager.io/v1, Kind=Certificate"}
--- PASS: TestReconcileCertManager (0.01s)
    --- PASS: TestReconcileCertManager/No_cert-manager_Objects (0.00s)
    --- PASS: TestReconcileCertManager/cert-manager_certificates_defined (0.00s)
    --- PASS: TestReconcileCertManager/cert-manager_crd_defined (0.00s)
PASS
coverage: 80.0% of statements
ok  	github.com/shipwright-io/operator/pkg/certmanager	0.055s	coverage: 80.0% of statements
=== RUN   TestImagesFromEnv
--- PASS: TestImagesFromEnv (0.00s)
=== RUN   TestDeploymentImages
=== RUN   TestDeploymentImages/ignore_non_deployment_images
=== RUN   TestDeploymentImages/replace_containers_image_by_name
=== RUN   TestDeploymentImages/replace_containers_env
--- PASS: TestDeploymentImages (0.01s)
    --- PASS: TestDeploymentImages/ignore_non_deployment_images (0.00s)
    --- PASS: TestDeploymentImages/replace_containers_image_by_name (0.01s)
    --- PASS: TestDeploymentImages/replace_containers_env (0.00s)
=== RUN   TestTruncateNestedFields
=== RUN   TestTruncateNestedFields/test_truncation_of_manifests
--- PASS: TestTruncateNestedFields (0.00s)
    --- PASS: TestTruncateNestedFields/test_truncation_of_manifests (0.00s)
=== RUN   TestTruncateCRDFieldTransformer
=== RUN   TestTruncateCRDFieldTransformer/test_truncate_CRD_field_Transformer
--- PASS: TestTruncateCRDFieldTransformer (0.00s)
    --- PASS: TestTruncateCRDFieldTransformer/test_truncate_CRD_field_Transformer (0.00s)
=== RUN   TestBoolFromEnvVar
--- PASS: TestBoolFromEnvVar (0.00s)
PASS
coverage: 42.7% of statements
ok  	github.com/shipwright-io/operator/pkg/common	0.060s	coverage: 42.7% of statements
=== RUN   TestReconcileTekton
=== RUN   TestReconcileTekton/No_Tekton_Objects
=== RUN   TestReconcileTekton/Tekton_TaskRuns_defined
=== RUN   TestReconcileTekton/TektonConfig_defined_no_version
=== RUN   TestReconcileTekton/TektonConfig_defined_insufficient_version
=== RUN   TestReconcileTekton/TektonConfig_defined_sufficient_version
=== RUN   TestReconcileTekton/Create_TektonConfig_error
=== RUN   TestReconcileTekton/TektonConfig_already_created
--- PASS: TestReconcileTekton (0.00s)
    --- PASS: TestReconcileTekton/No_Tekton_Objects (0.00s)
    --- PASS: TestReconcileTekton/Tekton_TaskRuns_defined (0.00s)
    --- PASS: TestReconcileTekton/TektonConfig_defined_no_version (0.00s)
    --- PASS: TestReconcileTekton/TektonConfig_defined_insufficient_version (0.00s)
    --- PASS: TestReconcileTekton/TektonConfig_defined_sufficient_version (0.00s)
    --- PASS: TestReconcileTekton/Create_TektonConfig_error (0.00s)
    --- PASS: TestReconcileTekton/TektonConfig_already_created (0.00s)
=== RUN   TestIsTektonPipelinesInstalled
=== RUN   TestIsTektonPipelinesInstalled/Not_present
=== RUN   TestIsTektonPipelinesInstalled/Tekton_present
=== RUN   TestIsTektonPipelinesInstalled/Error_getting_TaskRuns
--- PASS: TestIsTektonPipelinesInstalled (0.00s)
    --- PASS: TestIsTektonPipelinesInstalled/Not_present (0.00s)
    --- PASS: TestIsTektonPipelinesInstalled/Tekton_present (0.00s)
    --- PASS: TestIsTektonPipelinesInstalled/Error_getting_TaskRuns (0.00s)
=== RUN   TestIsTektonOperatorInstalled
=== RUN   TestIsTektonOperatorInstalled/Not_present
=== RUN   TestIsTektonOperatorInstalled/Tekton_Operator_present
=== RUN   TestIsTektonOperatorInstalled/Error_getting_TetkonConfigs
--- PASS: TestIsTektonOperatorInstalled (0.00s)
    --- PASS: TestIsTektonOperatorInstalled/Not_present (0.00s)
    --- PASS: TestIsTektonOperatorInstalled/Tekton_Operator_present (0.00s)
    --- PASS: TestIsTektonOperatorInstalled/Error_getting_TetkonConfigs (0.00s)
=== RUN   TestGetTektonOperatorVersion
=== RUN   TestGetTektonOperatorVersion/No_CRD
=== RUN   TestGetTektonOperatorVersion/Get_CRD_error
=== RUN   TestGetTektonOperatorVersion/No_labels_on_TektonConfig_CRD
=== RUN   TestGetTektonOperatorVersion/No_release_label_on_TektonConfig_CRD
=== RUN   TestGetTektonOperatorVersion/release_label_on_TektonConfig_CRD_is_not_a_semver
=== RUN   TestGetTektonOperatorVersion/Valid_TektonConfig_CRD
--- PASS: TestGetTektonOperatorVersion (0.00s)
    --- PASS: TestGetTektonOperatorVersion/No_CRD (0.00s)
    --- PASS: TestGetTektonOperatorVersion/Get_CRD_error (0.00s)
    --- PASS: TestGetTektonOperatorVersion/No_labels_on_TektonConfig_CRD (0.00s)
    --- PASS: TestGetTektonOperatorVersion/No_release_label_on_TektonConfig_CRD (0.00s)
    --- PASS: TestGetTektonOperatorVersion/release_label_on_TektonConfig_CRD_is_not_a_semver (0.00s)
    --- PASS: TestGetTektonOperatorVersion/Valid_TektonConfig_CRD (0.00s)
=== RUN   TestIsTektonConfigPresent
=== RUN   TestIsTektonConfigPresent/No_TektonConfig
=== RUN   TestIsTektonConfigPresent/List_TektonConfig_error
=== RUN   TestIsTektonConfigPresent/TektonConfig_present
--- PASS: TestIsTektonConfigPresent (0.00s)
    --- PASS: TestIsTektonConfigPresent/No_TektonConfig (0.00s)
    --- PASS: TestIsTektonConfigPresent/List_TektonConfig_error (0.00s)
    --- PASS: TestIsTektonConfigPresent/TektonConfig_present (0.00s)
=== RUN   TestCreateTektonConfig
--- PASS: TestCreateTektonConfig (0.00s)
PASS
coverage: 91.5% of statements
ok  	github.com/shipwright-io/operator/pkg/tekton	0.074s	coverage: 91.5% of statements
	github.com/shipwright-io/operator/test		coverage: 0.0% of statements
FAIL
make: *** [Makefile:152: test] Error 1

I'm not sure why this works now lol 🥲

Maybe we were just one workflow re-run away from figuring out it was a temporary download failure yesterday.

IrvingMg · 2026-03-02T09:47:46Z

I looked into the issue and was able to reproduce both problems.

The first one:

# golang.org/x/tools/internal/tokeninternal
../../../go/pkg/mod/golang.org/x/tools@v0.20.0/internal/tokeninternal/tokeninternal.go:64:9: invalid array length -delta * delta (constant -256 of type int64)
make: *** [Makefile:208: controller-gen] Error 1

This happens due to an incompatibility between golang.org/x/tools@v0.20.0 and newer Go versions. The operator uses Go 1.24.4.

If you run make test with Go 1.24.4, the controller-gen error no longer appears. However, you get the following error, which is the same as in CI:

unable to list versions to find latest one: unable to list bucket items -- got status "401 Unauthorized" from GCS

This occurs because looks like the bucket where older versions of the envtest binaries were stored has changed, and the required permissions are no longer available. Therefore, bumping the envtest version appears to be the correct fix.

For now, I suggest keeping the version bump limited to envtest. The operator still uses Go 1.24.4, and controller-gen does not seem to be an issue with that version. As a follow-up, we could consider bumping the Go version and other dependencies, but to keep this PR focused on the current issue, updating envtest should be sufficient.

Makefile

Signed-off-by: karthik balasubramanian <karthikbalasubramanian08@gmail.com>

IrvingMg

Thanks for the fix, @kaizakin!

/lgtm

It looks like #279 is a broader PR that bumps several other dependencies, including envtest. However, we might want to start by bumping envtest first, since the version currently used in the repo seems to be broken and requires a fix.

Let’s see what @hasanawad94 thinks, as he’s working on the dependency bumps in #279.

adambkaplan

/approve

@hasanawad94 FYI - I'm going to ✅ this pull request so that we unblock CI testing. You will need to rebase your PR and drop your envtest change when the inevitable merge conflict arrives.

openshift-ci · 2026-03-04T23:25:15Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: adambkaplan

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

~~OWNERS~~ [adambkaplan]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

pull-request-size bot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Feb 26, 2026

openshift-ci bot requested review from SaschaSchwarze0 and qu1queee February 26, 2026 20:29

openshift-ci bot added the do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. label Feb 26, 2026

shipwright-ci-bot added this to Issues Feb 26, 2026

IrvingMg reviewed Feb 27, 2026

View reviewed changes

IrvingMg reviewed Mar 2, 2026

View reviewed changes

Makefile Show resolved Hide resolved

chore: bump envtest version

117d852

Signed-off-by: karthik balasubramanian <karthikbalasubramanian08@gmail.com>

kaizakin force-pushed the feat/updatemakefile branch from 85df758 to 117d852 Compare March 2, 2026 11:45

pull-request-size bot added size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Mar 2, 2026

adambkaplan added this to the release-v0.19.0 milestone Mar 2, 2026

openshift-ci bot added release-note-none and removed do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. labels Mar 2, 2026

IrvingMg reviewed Mar 4, 2026

View reviewed changes

openshift-ci bot assigned IrvingMg Mar 4, 2026

openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Mar 4, 2026

adambkaplan approved these changes Mar 4, 2026

View reviewed changes

openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Mar 4, 2026

openshift-merge-bot bot merged commit 1102136 into shipwright-io:main Mar 4, 2026
5 checks passed

github-project-automation bot moved this to Done in Issues Mar 4, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

CHORE: bump controller-gen/envtest and set --use-deprecated-gcs in Makefile recipe to false #281

CHORE: bump controller-gen/envtest and set --use-deprecated-gcs in Makefile recipe to false #281
openshift-merge-bot[bot] merged 1 commit intoshipwright-io:mainfrom
kaizakin:feat/updatemakefile

kaizakin commented Feb 26, 2026 •

edited by adambkaplan

Loading

Uh oh!

kaizakin commented Feb 26, 2026

Uh oh!

IrvingMg left a comment

Uh oh!

kaizakin commented Feb 27, 2026

Uh oh!

IrvingMg commented Feb 27, 2026

Uh oh!

kaizakin commented Feb 27, 2026

Uh oh!

kaizakin commented Feb 27, 2026

Uh oh!

IrvingMg commented Mar 2, 2026

Uh oh!

Uh oh!

IrvingMg left a comment

Uh oh!

adambkaplan left a comment

Uh oh!

openshift-ci bot commented Mar 4, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

kaizakin commented Feb 26, 2026 • edited by adambkaplan Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

Submitter Checklist

Release Notes

Uh oh!

kaizakin commented Feb 26, 2026

Uh oh!

IrvingMg left a comment

Choose a reason for hiding this comment

Uh oh!

kaizakin commented Feb 27, 2026

Uh oh!

IrvingMg commented Feb 27, 2026

Uh oh!

kaizakin commented Feb 27, 2026

Uh oh!

kaizakin commented Feb 27, 2026

For context This was the error i was facing yesterday

Uh oh!

IrvingMg commented Mar 2, 2026

Uh oh!

Uh oh!

IrvingMg left a comment

Choose a reason for hiding this comment

Uh oh!

adambkaplan left a comment

Choose a reason for hiding this comment

Uh oh!

openshift-ci bot commented Mar 4, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

kaizakin commented Feb 26, 2026 •

edited by adambkaplan

Loading