Skip to content

feat(chart): opt in to shared control plane and CAS client endpoints #3726

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
migmartri wants to merge 1 commit into
base: main
Choose a base branch
from
+97 −39
Open

feat(chart): opt in to shared control plane and CAS client endpoints #3726

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion deployment/chainloop/Chart.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ description: Chainloop is an open source software supply chain control plane, a

type: application
# Bump the patch (not minor, not major) version on each change in the Chart Source code
version: 1.474.0
version: 1.474.1
# Do not update appVersion, this is handled automatically by the release process
appVersion: v1.127.4

Expand Down
71 changes: 38 additions & 33 deletions deployment/chainloop/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -241,6 +241,8 @@ or as shown before, provide them as imperative inputs during Helm Install/Upgrad

### Enable a custom domain with TLS

Since Chainloop v1.126.0 (chart version 1.469.0), the control plane and the Artifact CAS each serve native gRPC and HTTP on one port, exposed through the shared ingress (`controlplane.ingress` and `cas.ingress`). The separate gRPC-only endpoints (`serviceAPI` and `ingressAPI`) are deprecated but remain available. See [Opt in to shared client endpoints](#opt-in-to-shared-client-endpoints).

The control plane ingress serves HTTP, gRPC-Web, and native gRPC on the same hostname.
Its HTTP server routes each request, so these protocols can share an upstream HTTP/2 connection.
The existing gRPC ingress stays available. Existing clients can keep their current endpoints.
Expand All @@ -263,7 +265,7 @@ cas:

With `tls: true`, configure a certificate source — the cert-manager issuer annotation above, or a pre-created Secret named `cas.example.com-tls` — so the rendered Ingress references a Secret that exists.

`cas.ingress.enabled` defaults to `false`, so a setup that only copies the documented `cas.ingressAPI` block leaves the shared hostname disabled. Retain `cas.ingressAPI` only while the legacy gRPC endpoint is also required. `cas.serviceAPI`, and the control plane's connection to the legacy CAS endpoint keep their existing settings. CAS adds no gRPC-Web or Connect support. Deployed-controller qualification and the full streaming-performance evidence still need work. See the [experiment instructions](../../app/artifact-cas/README.md#shared-endpoint).
`cas.ingress.enabled` defaults to `false`, so a setup that only copies the documented `cas.ingressAPI` block leaves the shared hostname disabled. Retain `cas.ingressAPI` only while the legacy gRPC endpoint is also required. `cas.serviceAPI` keeps its existing settings. The control plane connects to the legacy CAS endpoint unless `useSharedEndpoints` is enabled. CAS adds no gRPC-Web or Connect support. Deployed-controller qualification and the full streaming-performance evidence still need work. See the [experiment instructions](../../app/artifact-cas/README.md#shared-endpoint).

The chart configures HTTP/2 upstream traffic for these controllers:

Expand Down Expand Up @@ -299,72 +301,74 @@ The NGINX gRPC module path supports the shared endpoint without path or content-
The regression suite checks that path with NGINX Open Source. It does not certify every controller or NGINX Plus release.
See the [F5 annotation reference](https://docs.nginx.com/nginx-ingress-controller/configuration/ingress-resources/advanced-configuration-with-annotations/) for controller requirements.

Enable the shared control plane ingress and the existing gRPC ingresses as needed.

See below an example of a `values.yaml` override

```yaml
useSharedEndpoints: true

controlplane:
ingress:
enabled: true
hostname: cp.chainloop.dev

ingressAPI:
enabled: true
hostname: api.cp.chainloop.dev

cas:
ingressAPI:
enabled: true
hostname: api.cas.chainloop.dev
ingress:
enabled: true
hostname: cas.chainloop.dev
```

A complete setup that uses

- [NGINX as ingress Controller](https://kubernetes.github.io/ingress-nginx)
- [cert-manager](https://cert-manager.io/) as TLS provider

would look like
would look like the following. The chart already sets the gRPC backend protocol and the CAS upload body-size annotations on the shared ingresses.

```yaml
useSharedEndpoints: true

controlplane:
ingress:
enabled: true
tls: true
ingressClassName: nginx
hostname: cp.chainloop.dev
annotations:
nginx.ingress.kubernetes.io/backend-protocol: "GRPC"
# This depends on your configured issuer
cert-manager.io/cluster-issuer: "letsencrypt-prod"

ingressAPI:
enabled: true
tls: true
ingressClassName: nginx
hostname: api.cp.chainloop.dev
annotations:
nginx.ingress.kubernetes.io/backend-protocol: "GRPC"
cert-manager.io/cluster-issuer: "letsencrypt-prod"

cas:
ingressAPI:
ingress:
enabled: true
tls: true
ingressClassName: nginx
hostname: api.cas.chainloop.dev
hostname: cas.chainloop.dev
annotations:
nginx.ingress.kubernetes.io/backend-protocol: "GRPC"
cert-manager.io/cluster-issuer: "letsencrypt-prod"
# limit the size of the files that go through the proxy
# 0 means to not check the size of the request so we do not get 413 error.
# For now we are going to set a limit on 100MB files
# Even though we send data in chunks of 1MB, this size refers to all the data sent in the streaming connection
nginx.ingress.kubernetes.io/proxy-body-size: "100m"
```

Remember, once you have set up your domain, make sure you use the [CLI pointing](#configure-chainloop-cli-to-point-to-your-instance) to it instead of the defaults.

#### Opt in to shared client endpoints

The chart keeps the existing client endpoint selection by default. Enable `useSharedEndpoints` to move native gRPC clients to the shared HTTP endpoints:

```yaml
useSharedEndpoints: true
```

This flag switches the control plane CAS client to the shared CAS Service. It also changes the CLI addresses in the installation notes to the shared ingress hostnames. With the examples above, the CLI uses `cp.chainloop.dev:443` and `cas.chainloop.dev:443`.

The separate control plane and CAS `ingressAPI` and `serviceAPI` settings are deprecated. They remain functional in this release. The flag changes client selection, not endpoint exposure. Removal belongs to a later release after existing clients migrate.

An installed CLI retains its saved addresses. Reconfigure it explicitly when migrating:

```bash
chainloop config save --control-plane cp.chainloop.dev:443 --artifact-cas cas.chainloop.dev:443
```

Set `useSharedEndpoints: false` to restore the previous client endpoint selection. Reconfigure installed CLIs separately if they need to return to the legacy addresses.

### Connect to an external PostgreSQL database

```yaml
Expand Down Expand Up @@ -581,6 +585,7 @@ Once done, you can access with [two predefined users](https://github.com/chainlo
| `global.imagePullSecrets` | Global Docker registry secret names as an array | `[]` |
| `global.compatibility.openshift.adaptSecurityContext` | Adapt the securityContext sections of the deployment to make them compatible with Openshift restricted-v2 SCC: remove runAsUser, runAsGroup and fsGroup and let the platform use their allowed default IDs. Possible values: auto (apply if the detected running cluster is Openshift), force (perform the adaptation always), disabled (do not perform adaptation) | `auto` |
| `development` | Deploys Chainloop pre-configured FOR DEVELOPMENT ONLY. It includes a Vault instance in development mode and pre-configured authentication certificates and passphrases | `false` |
| `useSharedEndpoints` | Opt in to shared control plane and CAS HTTP endpoints for native gRPC clients and the CLI addresses in the installation notes. Legacy endpoints remain available. | `false` |

### Common parameters

Expand Down Expand Up @@ -697,7 +702,7 @@ Once done, you can access with [two predefined users](https://github.com/chainlo
| `controlplane.service.extraPorts` | Extra ports to expose in controlplane service (normally used with the `sidecars` value) | `[]` |
| `controlplane.service.sessionAffinity` | Control where client requests go, to the same pod or round-robin | `None` |
| `controlplane.service.sessionAffinityConfig` | Additional settings for the sessionAffinity | `{}` |
| `controlplane.serviceAPI.type` | Service type | `ClusterIP` |
| `controlplane.serviceAPI.type` | Deprecated gRPC-only Service type. Use the shared service with useSharedEndpoints enabled. | `ClusterIP` |
| `controlplane.serviceAPI.ports.http` | controlplane service HTTP port | `80` |
| `controlplane.serviceAPI.ports.https` | controlplane service HTTPS port | `443` |
| `controlplane.serviceAPI.nodePorts.http` | Node port for HTTP | `""` |
Expand All @@ -723,7 +728,7 @@ Once done, you can access with [two predefined users](https://github.com/chainlo
| `controlplane.ingress.extraTls` | TLS configuration for additional hostname(s) to be covered with this ingress record | `[]` |
| `controlplane.ingress.secrets` | Custom TLS certificates as secrets | `[]` |
| `controlplane.ingress.extraRules` | Additional rules to be covered with this ingress record | `[]` |
| `controlplane.ingressAPI.enabled` | Enable ingress record generation for controlplane | `false` |
| `controlplane.ingressAPI.enabled` | Enable the deprecated gRPC-only ingress. Retained during migration. | `false` |
| `controlplane.ingressAPI.pathType` | Ingress path type | `ImplementationSpecific` |
| `controlplane.ingressAPI.hostname` | Default host for the ingress record | `api.cp.dev.local` |
| `controlplane.ingressAPI.ingressClassName` | IngressClass that will be be used to implement the Ingress (Kubernetes 1.18+) | `""` |
Expand Down Expand Up @@ -879,7 +884,7 @@ Once done, you can access with [two predefined users](https://github.com/chainlo
| `cas.service.extraPorts` | Extra ports to expose in cas service (normally used with the `sidecars` value) | `[]` |
| `cas.service.sessionAffinity` | Control where client requests go, to the same pod or round-robin | `None` |
| `cas.service.sessionAffinityConfig` | Additional settings for the sessionAffinity | `{}` |
| `cas.serviceAPI.type` | Service type | `ClusterIP` |
| `cas.serviceAPI.type` | Deprecated gRPC-only Service type. Use the shared service with useSharedEndpoints enabled. | `ClusterIP` |
| `cas.serviceAPI.ports.http` | cas service HTTP port | `80` |
| `cas.serviceAPI.ports.https` | cas service HTTPS port | `443` |
| `cas.serviceAPI.nodePorts.http` | Node port for HTTP | `""` |
Expand All @@ -905,7 +910,7 @@ Once done, you can access with [two predefined users](https://github.com/chainlo
| `cas.ingress.extraTls` | TLS configuration for additional hostname(s) to be covered with this ingress record | `[]` |
| `cas.ingress.secrets` | Custom TLS certificates as secrets | `[]` |
| `cas.ingress.extraRules` | Additional rules to be covered with this ingress record | `[]` |
| `cas.ingressAPI.enabled` | Enable ingress record generation for controlplane | `false` |
| `cas.ingressAPI.enabled` | Enable the deprecated gRPC-only ingress. Retained during migration. | `false` |
| `cas.ingressAPI.pathType` | Ingress path type | `ImplementationSpecific` |
| `cas.ingressAPI.hostname` | Default host for the ingress record | `api.cas.dev.local` |
| `cas.ingressAPI.ingressClassName` | IngressClass that will be be used to implement the Ingress (Kubernetes 1.18+) | `""` |
Expand Down
30 changes: 30 additions & 0 deletions deployment/chainloop/chart_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -277,6 +277,36 @@ type chartProbe struct {
} `yaml:"httpGet"`
}

// useSharedEndpoints moves the control plane CAS client to the shared CAS
// Service. The legacy gRPC Services and ingresses remain available.
func TestUseSharedEndpoints(t *testing.T) {
for _, tc := range []struct {
name string
args []string
casAddr string
}{
{name: "defaults", casAddr: "test-chainloop-cas-api:80"},
{name: "explicitly disabled", args: []string{helmSet, "useSharedEndpoints=false"}, casAddr: "test-chainloop-cas-api:80"},
{name: "enabled", args: []string{helmSet, "useSharedEndpoints=true"}, casAddr: "test-chainloop-cas:80"},
{
name: "enabled with custom service port",
args: []string{helmSet, "useSharedEndpoints=true", helmSet, "cas.service.port=8083"},
casAddr: "test-chainloop-cas:8083",
},
} {
t.Run(tc.name, func(t *testing.T) {
resources := renderChart(t, tc.args)
require.Contains(t, resources["ConfigMap/test-chainloop-controlplane"].Data["config.yaml"], "addr: "+tc.casAddr+"\n")
for _, name := range []string{"test-chainloop-controlplane-api", "test-chainloop-cas-api"} {
_, ok := resources["Service/"+name]
require.True(t, ok, name)
_, ok = resources["Ingress/"+name]
require.True(t, ok, name)
}
})
}
}

func TestCASSharedIngressTLS(t *testing.T) {
resources := renderChart(t, []string{helmSet, casIngressEnabled, helmSet, "cas.ingress.tls=true",
helmSetString, `cas.ingress.annotations.cert-manager\.io/cluster-issuer=test-issuer`})
Expand Down
19 changes: 19 additions & 0 deletions deployment/chainloop/templates/_helpers.tpl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -337,6 +337,10 @@ NOTE: Load balancer service type is not supported
{{- define "chainloop.controlplane.external_grpc_url" -}}
{{- $service := .Values.controlplane.serviceAPI }}
{{- $ingress := .Values.controlplane.ingressAPI }}
{{- if .Values.useSharedEndpoints }}
{{- $service = .Values.controlplane.service }}
{{- $ingress = .Values.controlplane.ingress }}
{{- end }}

{{- if (and $ingress $ingress.enabled $ingress.hostname) }}
{{- printf "%s:%s" $ingress.hostname (ternary "443" "80" $ingress.tls) }}
Expand Down Expand Up @@ -439,9 +443,24 @@ External gRPC URL the CAS API can be reached at
This endpoint is used by the CLI to connect via gRPC
NOTE: Load balancer service type is not supported
*/}}
{{/*
Artifact CAS gRPC address used by the control plane (host:port)
*/}}
{{- define "chainloop.cas.internal_url" -}}
{{- if .Values.useSharedEndpoints }}
{{- printf "%s:%d" (include "chainloop.cas.fullname" .) (coalesce .Values.cas.service.port .Values.cas.service.ports.http | int) }}
{{- else }}
{{- printf "%s-api:%d" (include "chainloop.cas.fullname" .) (.Values.cas.serviceAPI.ports.http | int) }}
{{- end }}
{{- end -}}

{{- define "chainloop.cas.external_grpc_url" -}}
{{- $service := .Values.cas.serviceAPI }}
{{- $ingress := .Values.cas.ingressAPI }}
{{- if .Values.useSharedEndpoints }}
{{- $service = .Values.cas.service }}
{{- $ingress = .Values.cas.ingress }}
{{- end }}

{{- if (and $ingress $ingress.enabled $ingress.hostname) }}
{{- printf "%s:%s" $ingress.hostname (ternary "443" "80" $ingress.tls) }}
Expand Down
2 changes: 1 addition & 1 deletion deployment/chainloop/templates/controlplane/configmap.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ data:
{{- end }}
cas_server:
grpc:
addr: {{ printf "%s-api:%.0f" (include "chainloop.cas.fullname" .) .Values.cas.serviceAPI.ports.http }}
addr: {{ include "chainloop.cas.internal_url" . }}
insecure: {{ empty (include "controlplane.tls-secret-name" .) }}
download_url: {{ include "chainloop.cas.external_url" . }}/download
{{- if .Values.cas.defaultMaxEntrySize }}
Expand Down
Loading
Loading