design-proposal: structured, additive external exposure for managed applications

[Andrei Kvapil (kvaps)]

What

Adds a design proposal under design-proposals/structured-external-exposure/ that replaces the chart-level boolean external: true|false on managed applications with a structured, additive expose field — a list of {target, class} entries.

Why

The boolean is a dead end: it publishes exactly one endpoint, offers no way to choose the address source, cannot select which named listener of an engine is published, and reports its result by mutating the in-cluster connection details in place. Each chart also implements it slightly differently, so there is no shared model or status contract.

The model

• expose: [] — a list of {target, class} entries. Absent or empty means in-cluster only. The in-cluster ClusterIP Service is always present and never touched; every entry is additive.
• target — a per-engine closed enum derived from the engine's existing named listeners (Postgres rw/ro, ClickHouse http/tcp, and so on); omitted selects the engine's primary listener.
• class — a reference to an admin-defined ExposureClass (StorageClass-like, #3081); omitted selects the cluster default. Each entry renders an additive ClusterIP Service plus a ServiceExposure, and the controller allocates the external address and reports it.
• status.endpoints[] — {target, scope, host, port, tls}, resolved from ServiceExposure.status.assignedIPs, never fabricated client-side.
• TLS — external exposure implies TLS, terminated at the database pod, validated against the merged ca.crt-only trust anchor (#2989).
• Source-IP ACL is out of the field — that is SecurityGroup (#2922), with the deny/allow posture owned by the consuming orchestrator, not Cozystack core.
• external: true stays accepted as a deprecated alias for a single primary entry.

Scope

Cozystack side only — the chart API, the rendered objects, and the reported status. It builds entirely on already-merged primitives (#3081, #2989, #2922, #2470) and stays forward-compatible with future Gateway/SNI endpoint consolidation (community #20) without implementing it.

Status

Draft — opening for early design feedback before the API shape is locked.

[coderabbitai]

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 49525166-96d5-484b-b210-3d4f270a4212

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch design/structured-external-exposure

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[gemini-code-assist]

Code Review

This pull request introduces a design proposal for structured, additive external exposure for managed applications in Cozystack, replacing the binary external boolean with a flexible expose list of {target, class} entries. The feedback highlights several important areas for improvement: expanding the target list for MariaDB, MongoDB, and Redis to support read-replicas; including the actual class name in the endpoint status for better traceability; and addressing a critical TLS verification issue where dynamically allocated external IPs will not match the certificate SANs when TLS is terminated directly at the database pod.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

[gemini-code-assist]

For databases like MariaDB, MongoDB, and Redis, it is highly common to route read-only traffic to replicas or secondaries to scale read performance. Restricting these engines to only a primary target prevents users from exposing read-replicas externally.

Consider expanding the target list to support replicas, for example:

• mariadb: primary, replica
• mongodb: primary, secondary
• redis: master, replica

[gemini-code-assist]

Since ExposureClass names are arbitrary and defined by the cluster administrator (e.g., dmz, office-vpn, public-direct), relying solely on a fixed scope enum (in-cluster, public, private) in status.endpoints[] makes it difficult for consumers to map an endpoint back to its specific ExposureClass. This is especially problematic if there are multiple classes with the same scope or if the scope mapping is not obvious.

Including the actual class (or exposureClass) name in the endpoint status would provide clear traceability.

For example, the endpoint entry could be:

    - target: rw
      scope: private
      class: private-vpn
      host: 10.0.0.5
      port: 5432
      tls: true

[gemini-code-assist]

Terminating TLS directly at the database pod introduces a significant challenge regarding certificate SANs (Subject Alternative Names).

Since the external IP addresses (e.g., 10.0.0.5, 203.0.113.7) are allocated dynamically by the serviceexposure-controller after the database pods are already running, the certificates initially minted for the pods will not contain these external IPs in their SANs. If an external client attempts to connect to these IPs, TLS hostname/IP verification will fail unless the certificates are dynamically re-issued and reloaded by the database engine.

Not all database engines support hot-reloading TLS certificates without a restart, and frequent restarts upon IP allocation/re-allocation are undesirable.

It would be highly beneficial to address how SANs for dynamically allocated IPs/hosts will be handled, or if a DNS-based approach (where the certificate uses a wildcard or stable DNS SAN, and external clients connect via DNS rather than raw IPs) is expected to mitigate this.

[myasnikovdaniil]

The field shape (expose: [{target, class}]) and the additive-status goal are the right direction, but the design's central mechanism doesn't match how exposure actually works across the managed apps, and the engine table has factual errors. I went through every app under cozystack/cozystack packages/apps/. Three problems, two of them structural:

1. "The chart renders one additive type: LoadBalancer Service per target" is false for most engines. Only postgres, redis, opensearch, and mongodb's sharded-mongos path have a chart-owned Service. For kafka, rabbitmq, mariadb, qdrant, nats, and mongodb's replica-set path the chart only flips a field on an operator CR or subchart values — the operator/subchart owns Service creation. A Helm render reading Service.status.loadBalancer.ingress back can't even attribute which Services it should read. The mechanism section, the mermaid diagram, and the unit-test plan all assume a chart-owned Service that mostly doesn't exist.

2. The advertised-address feedback loop is missing for the discovery engines, and the design removed the only component that could host it. For kafka (Strimzi writes advertised.listeners + rolls), mongodb replica sets (Percona creates one LB per pod and rewrites rs.conf, issue #2514), and nats (connect_urls gossip), the allocated external address must be written back into the engine config and the pods rolled — a loop a Helm render fundamentally cannot do. Today the per-engine operator owns it. This proposal never mentions it, and explicitly removes the ServiceExposure controller (#3164) that was the natural home for it. So for the discovery engines the design either silently delegates to the operator (then say so, and drop "the chart renders the Service") or it has no owner.

3. Factual errors in the engine table. clickhouse and bucket have no external field today — there is no existing external listener to "make selectable", so those rows are net-new exposure work, which contradicts "this proposal does not add new listeners." And the redis app is Sentinel-based redis-failover fronting the master directly — not Redis Cluster — so it has no cluster-announce-ip, and the primary row conflates a Class-B engine (mongodb) with two Class-A ones (mariadb, redis).

What I'd want before this is mergeable: (a) split the engines into a passthrough class (single endpoint, report the IP — postgres/mariadb/rabbitmq/redis/opensearch/qdrant) and a discovery class (advertised address + per-peer fan-out + roll — kafka/mongodb-RS/nats), and design each explicitly rather than as one uniform mechanism; (b) name who owns the read-back-and-reconfigure loop for the discovery class (operator delegation vs a controller), given the boolean's own controller was removed; (c) correct the table (clickhouse/bucket = new capability; redis = Sentinel not Cluster; split the primary row). Inline detail on each spot below.

[myasnikovdaniil]

bootstrap as a single target doesn't model Kafka external access. external: true today flips a Strimzi listener type: loadbalancer (packages/apps/kafka/templates/kafka.yaml:37-42); Strimzi then creates a bootstrap LB plus one LB per broker, reads the allocated IPs, writes them into each broker's advertised.listeners, and rolls the pods. So (a) it isn't one Service, it's a bootstrap + N-broker fan-out, and (b) the reachable set isn't a single bootstrap host — a client that only has bootstrap is redirected to per-broker addresses that must themselves be externally advertised. Neither the per-broker fan-out nor the advertised-address write-back is expressible in {target, class} → one Service.

[myasnikovdaniil]

This row collapses three engines that behave differently, and one of them is a Class-B discovery engine the uniform model can't handle:

• mongodb — the replica-set path flips replsets[].expose on the Percona CR (packages/apps/mongodb/templates/mongodb.yaml:106-113); the operator creates one LB per pod and rewrites rs.conf so external drivers can discover the primary. The in-repo comment says exactly this (packages/apps/mongodb/templates/external-svc.yaml:1-8, issue #2514). Per-replica fan-out + config write-back + roll — a single primary target hides all of it.
• redis — this is Sentinel-based redis-failover, not Redis Cluster (redisfailover.yaml type: sentinel; Service selector redisfailovers-role: master, redis/templates/service.yaml:22). No cluster-announce-ip; the LB fronts the master directly. Class A, but for a different reason than the table implies.
• mariadb — operator flips primaryService.type: LoadBalancer (packages/apps/mariadb/templates/mariadb.yaml:87-90), single primary Service, no redirect. Class A.

Folding B (mongodb) together with A (mariadb, redis) under one primary row is exactly where the uniform model buries the hard case.

[myasnikovdaniil]

"The set is exactly what each engine can already expose … does not add new listeners" doesn't hold for two rows in the table:

• clickhouse — there is no external key in the chart at all (packages/apps/clickhouse/ renders only a ClusterIP svc-template); the http/tcp targets are net-new exposure, not existing listeners made selectable.
• bucket — no external key either; S3 reach is via the platform's shared ingress, not a per-app LB. s3 "always public; special-cased" is new behavior.

These should be framed and designed as new capability, not as a refactor of an existing boolean.

[myasnikovdaniil]

This is the load-bearing claim of the design, and it's false for most engines in the table. Chart-owned hand-written Service: only postgres (packages/apps/postgres/templates/external-svc.yaml), redis (redis/templates/service.yaml), opensearch (opensearch/templates/external-svc.yaml), and mongodb's sharded-mongos path. For kafka, rabbitmq, mariadb, qdrant, nats, and mongodb's replica-set path the chart only flips a field on an operator CR (Strimzi / RabbitmqCluster / MariaDB / Percona) or on subchart values (nats, qdrant) — the operator or subchart creates the Service(s).

Two consequences: class → Service.spec.loadBalancerClass has to be threaded through each operator's own listener/service config, not set on a chart Service; and a render that "reads Service.status.loadBalancer.ingress back" can't know which operator-created Services to read. The mechanism, the mermaid diagram, and the unit-test plan ("expose entries render the expected additive Services") all assume an ownership model that doesn't hold.

[myasnikovdaniil]

The address flows one way here — out of Service.status into status.endpoints[] for reporting. For the discovery engines it must also flow the other way: the allocated address has to be written into the engine config and the pods rolled, or the reported endpoint is reachable at the first hop and dead on the redirect (kafka advertised.listeners; mongodb rs.conf, see mongodb/templates/external-svc.yaml:1-8; nats connect_urls). That write-back-and-roll step is absent.

Separately: status.endpoints[] is entirely new machinery — no chart reads Service.status.loadBalancer.ingress back today — so the reporting half is unbuilt for every engine, not only the hard ones. Worth stating what component populates it.

[myasnikovdaniil]

Pod-terminated TLS + a dynamically allocated address is the same chicken-and-egg as advertised.listeners: the address must land in the cert SAN, i.e. be known before the cert is minted. This is already engine-specific, not uniform — postgres appends a DNS name (not the dynamic IP) to spec.certificates.serverAltDNSNames (packages/apps/postgres/templates/db.yaml:23-26), and Strimzi bakes per-broker advertised hosts into the broker certs after allocation. With cert issuance declared a non-goal (line 47), the design needs to state which component closes the SAN gap per engine; it can't be a single uniform rule. (Gemini's automated review flagged this independently.)

[myasnikovdaniil]

Removing the ServiceExposure controller is the crux of problems 1 and 2 in the summary. A controller is the natural (and only) home for the allocate → read-back → reconfigure → roll loop, and for reading operator-created Service status back into status.endpoints[], because it can watch and act. A Helm render can't wait for the LB IP and then re-render the CR/StatefulSet with the advertised address baked in.

Dropping the controller doesn't make that work disappear — it pushes it onto the per-engine operators for the discovery engines (unstated), and leaves the chart-owned engines' status read-back with no owner at all. "The chart already owns the Service" is true for a minority; for the rest, this removed the component that made the model implementable.