Hi, awesome that crossplane project has been added and cant wait to try it out more, but unfortunately it is currently unusable for people who develop in devcontainers and in a closed company networks.
I've done some digging and i believe i would have to add 3 functionalities for this to work seamlessly.
- Configuring internal kubeconfig
Currently the crossplane project command only exports the kubeconfig with the external addresses:
https://github.com/crossplane/cli/blob/main/internal/project/controlplane/controlplane.go#L419
This could be fixed by adding an option to configure the export of the KinD kubeconfig with an --internal flag to use the internal addresses which are reachable inside devcontainer.
This fixes the following error on startup:
crossplane: error: failed to create crossplane-system namespace: failed to get server groups: Get "https://127.0.0.1:35341/api": dial tcp 127.0.0.1:35341: connect: connection refused
- Configuring kind docker network
While this solves the initial connectivity issue from a devcontainer, the cluster creation will still not work seamlessly as the "kind" docker network is automatically created. And this docker network does not automatically have connectivity to the dev-container.
A simple solution for this is that the user is allowed to specify the docker-network that kind should use when starting the up the containers. Perhaps with a --docker-network="" flag?
But KinD does not expose this as a configuration option in the go-library.
The only solution that i have found is to set the environment variable KIND_EXPERIMENTAL_DOCKER_NETWORK.
I quickly tested to set this environment-variable before the cluster creation before this line and it worked:
os.Setenv("KIND_EXPERIMENTAL_DOCKER_NETWORK", "the-network-my-devcontainer-is-in")
- Configure containerd registry mirrors
The last hurdle to overcome i believe, is configuring the containerd registry mirrors so that KinD fetches images from the configured registry-proxies.
In closed network environments, it is usually required that container-images are fetched via a central registry, so for example docker.io cannot be reached directly.
With the assumption that if a user passes ImageConfigs via the --init-resources flag, the user also most likely would like to configure the containerd registry mirrors, it seems like the best option is to parse the ImageConfigs the user supplies and configure the containerd with the containerdConfigPatches option in KinD like this:
containerdConfigPatches:
- |
[plugins."io.containerd.grpc.v1.cri".registry.mirrors]
[plugins."io.containerd.grpc.v1.cri".registry.mirrors."docker.io"]
endpoint = ["https://docker-remote.my-registry-proxy.com"]
This could be done in the existing createKindClusterConfig function.
What do you think? I'd be happy to send in a PR and test it more, but is this something you want to add support for?
Let me know if you have any questions!
What problem are you facing?
Running crossplane project in a dev-container and closed company network.
Hi, awesome that
crossplane projecthas been added and cant wait to try it out more, but unfortunately it is currently unusable for people who develop in devcontainers and in a closed company networks.I've done some digging and i believe i would have to add 3 functionalities for this to work seamlessly.
Currently the
crossplane projectcommand only exports the kubeconfig with the external addresses:https://github.com/crossplane/cli/blob/main/internal/project/controlplane/controlplane.go#L419
This could be fixed by adding an option to configure the export of the KinD kubeconfig with an
--internalflag to use the internal addresses which are reachable inside devcontainer.This fixes the following error on startup:
crossplane: error: failed to create crossplane-system namespace: failed to get server groups: Get "https://127.0.0.1:35341/api": dial tcp 127.0.0.1:35341: connect: connection refusedWhile this solves the initial connectivity issue from a devcontainer, the cluster creation will still not work seamlessly as the "kind" docker network is automatically created. And this docker network does not automatically have connectivity to the dev-container.
A simple solution for this is that the user is allowed to specify the docker-network that kind should use when starting the up the containers. Perhaps with a
--docker-network=""flag?But KinD does not expose this as a configuration option in the go-library.
The only solution that i have found is to set the environment variable
KIND_EXPERIMENTAL_DOCKER_NETWORK.I quickly tested to set this environment-variable before the cluster creation before this line and it worked:
The last hurdle to overcome i believe, is configuring the containerd registry mirrors so that KinD fetches images from the configured registry-proxies.
In closed network environments, it is usually required that container-images are fetched via a central registry, so for example docker.io cannot be reached directly.
With the assumption that if a user passes ImageConfigs via the
--init-resourcesflag, the user also most likely would like to configure the containerd registry mirrors, it seems like the best option is to parse theImageConfigsthe user supplies and configure the containerd with thecontainerdConfigPatchesoption in KinD like this:This could be done in the existing createKindClusterConfig function.
What do you think? I'd be happy to send in a PR and test it more, but is this something you want to add support for?
Let me know if you have any questions!
What problem are you facing?
Running
crossplane projectin a dev-container and closed company network.