Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 11 additions & 31 deletions docs/getting-started.md
Original file line number Diff line number Diff line change
Expand Up @@ -334,41 +334,21 @@ What exactly fails over, and what was measured, is in [High availability](archit

### Kubernetes: DNS automation for Gateway API resources

For Gateway API resources (for example Envoy Gateway with `Gateway` + `HTTPRoute`), use DNS records directly via `stackit_dns_record_set` until native provider support for `extensions.dns.gatewayApi` is available.

For the existing sample content in this repository (`landing_zone_sample_gateway` + `landing_zone_sample_http_route` in `src/_landing-zone-kubernetes.tf`), the DNS record is created automatically based on the Envoy Gateway LoadBalancer endpoint discovered via `kubernetes_resources`.

Implementation pattern:
Native `extensions.dns.gateway_api` support is available in the STACKIT Terraform provider since v0.109.0 and is enabled by default whenever the SKE DNS extension has at least one zone. Disable it explicitly if the cluster does not use Gateway API resources:

```hcl
# Discover Envoy-managed LoadBalancer service endpoint for each sample gateway
data "kubernetes_resources" "landing_zone_sample_gateway_service" {
provider = kubernetes.platform

api_version = "v1"
kind = "Service"
namespace = "envoy-gateway-system"
label_selector = "gateway.envoyproxy.io/owning-gateway-name=<gateway-name>,gateway.envoyproxy.io/owning-gateway-namespace=<namespace>"
}

# Create A or CNAME record depending on endpoint type
resource "stackit_dns_record_set" "landing_zone_sample_gateway" {
project_id = module.landing_zone["corp-exmpl"].project_id
zone_id = module.landing_zone["corp-exmpl"].dns_zone_id

name = "app.${module.landing_zone["corp-exmpl"].dns_zone_dns_name}"
type = local.endpoint.ip != null ? "A" : "CNAME"
ttl = 60

records = [coalesce(local.endpoint.ip, local.endpoint.hostname)]

lifecycle {
precondition {
condition = local.endpoint.ip != null || local.endpoint.hostname != null
error_message = "Gateway load balancer endpoint is not available yet for DNS record creation."
platform_kubernetes = {
my-cluster = {
dns = {
enabled = true
gateway_api = false
}
# ...
}
}
```

This ensures a stable, Terraform-managed DNS path without external scripts until provider-native `gatewayApi` DNS extension support is available.
With the default `gateway_api = true`, the SKE DNS extension configures ExternalDNS to handle Gateway API resources (`Gateway`, `HTTPRoute`) automatically. You no longer need to manage DNS records or discover LoadBalancer endpoints manually.

> [!NOTE]
> The SKE DNS extension does not install Gateway API CRDs. The optional Envoy Gateway demo installs them automatically. For other Gateway API implementations, install compatible CRDs separately. Once the CRDs are available, ExternalDNS is configured during the next cluster reconcile; no additional OpenTofu apply is required.
5 changes: 1 addition & 4 deletions src/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,6 @@
| ---- | ------- |
| <a name="provider_helm.platform"></a> [helm.platform](#provider\_helm.platform) | 3.3.0 |
| <a name="provider_kubernetes.platform"></a> [kubernetes.platform](#provider\_kubernetes.platform) | 3.2.1 |
| <a name="provider_stackit"></a> [stackit](#provider\_stackit) | 0.114.0 |
| <a name="provider_terraform"></a> [terraform](#provider\_terraform) | n/a |
| <a name="provider_vault"></a> [vault](#provider\_vault) | 5.11.0 |

Expand Down Expand Up @@ -54,10 +53,8 @@
| [kubernetes_secret_v1.landing_zone_user_token](https://registry.terraform.io/providers/hashicorp/kubernetes/3.2.1/docs/resources/secret_v1) | resource |
| [kubernetes_service_account_v1.landing_zone_user](https://registry.terraform.io/providers/hashicorp/kubernetes/3.2.1/docs/resources/service_account_v1) | resource |
| [kubernetes_service_v1.landing_zone_sample_load](https://registry.terraform.io/providers/hashicorp/kubernetes/3.2.1/docs/resources/service_v1) | resource |
| [stackit_dns_record_set.landing_zone_sample_gateway](https://registry.terraform.io/providers/stackitcloud/stackit/0.114.0/docs/resources/dns_record_set) | resource |
| [terraform_data.firewall_api_bootstrap](https://registry.terraform.io/providers/hashicorp/terraform/latest/docs/resources/data) | resource |
| [vault_kv_secret_v2.firewall_api_credentials](https://registry.terraform.io/providers/hashicorp/vault/5.11.0/docs/resources/kv_secret_v2) | resource |
| [kubernetes_resources.landing_zone_sample_gateway_service](https://registry.terraform.io/providers/hashicorp/kubernetes/3.2.1/docs/data-sources/resources) | data source |

## Inputs

Expand All @@ -83,7 +80,7 @@
| <a name="input_organization_id"></a> [organization\_id](#input\_organization\_id) | Container ID of the root organization. | `string` | n/a | yes |
| <a name="input_organization_owners"></a> [organization\_owners](#input\_organization\_owners) | List of organization owners. | `list(string)` | `[]` | no |
| <a name="input_owner_email"></a> [owner\_email](#input\_owner\_email) | Email address of the owner. Required for STACKIT resource manager. | `string` | n/a | yes |
| <a name="input_platform_kubernetes"></a> [platform\_kubernetes](#input\_platform\_kubernetes) | Map of central, region-scoped platform Kubernetes deployments. Empty map skips deployment. | <pre>map(object({<br/> region = string<br/> network = optional(object({<br/> sna_enabled = optional(bool, false)<br/> sna_network_area_id = optional(string, null)<br/> firewall_next_hop_ip = optional(string, null)<br/> sna_network_prefix_length = optional(number, 24)<br/> }), {})<br/> dns = optional(object({<br/> enabled = optional(bool, true)<br/> create_zones = optional(bool, true)<br/> zones = optional(list(string), [])<br/> }), {})<br/> observability = optional(object({<br/> enabled = optional(bool, true)<br/> plan_name = optional(string, "Observability-Starter-EU01")<br/> acl = optional(list(string), [])<br/> name = optional(string, null)<br/> }), {})<br/> encrypted_volumes = optional(object({<br/> enabled = optional(bool, false)<br/> storage_class_name = optional(string, "stackit-encrypted-premium")<br/> kms_keyring_name = optional(string, "ske-volume-keyring")<br/> kms_key_name = optional(string, "ske-volume-key")<br/> kms_key_version = optional(string, "1")<br/> }), {})<br/> debug_bastion = optional(object({<br/> enabled = optional(bool, false)<br/> name = optional(string, null)<br/> availability_zone = optional(string, null)<br/> machine_type = optional(string, "g2i.1")<br/> image_id = optional(string, "7b10e105-295b-4369-b6e0-567ec940a02b")<br/> boot_volume_size = optional(number, 20)<br/> ssh_public_key = optional(string, null)<br/> ssh_public_key_path = optional(string, "~/.ssh/id_rsa.pub")<br/> ssh_allowed_cidrs = optional(list(string), ["0.0.0.0/0"])<br/> assign_public_ip = optional(bool, true)<br/> install_kubectl = optional(bool, true)<br/> }), {})<br/> role_assignments = optional(list(object({<br/> role = string<br/> subject = string<br/> })), [])<br/> cluster = object({<br/> name = string<br/> kubernetes_version_min = optional(string, null)<br/> node_pools = optional(list(object({<br/> name = string<br/> machine_type = string<br/> minimum = number<br/> maximum = number<br/> availability_zones = list(string)<br/> allow_system_components = optional(bool, false)<br/> volume_size = optional(number, 20)<br/> volume_type = optional(string, "storage_premium_perf1")<br/> os_name = optional(string, "flatcar")<br/> labels = optional(map(string), {})<br/> })), [<br/> {<br/> name = "system"<br/> machine_type = "g3i.4"<br/> minimum = 2<br/> maximum = 2<br/> availability_zones = ["eu01-1"]<br/> allow_system_components = true<br/> volume_size = 20<br/> volume_type = "storage_premium_perf1"<br/> os_name = "flatcar"<br/> labels = {<br/> "workload-role" = "system"<br/> }<br/> },<br/> {<br/> name = "application"<br/> machine_type = "g3i.4"<br/> minimum = 2<br/> maximum = 2<br/> availability_zones = ["eu01-2"]<br/> allow_system_components = false<br/> volume_size = 20<br/> volume_type = "storage_premium_perf1"<br/> os_name = "flatcar"<br/> labels = {<br/> "workload-role" = "application"<br/> }<br/> }<br/> ])<br/> maintenance = optional(object({<br/> enable_kubernetes_version_updates = optional(bool, true)<br/> enable_machine_image_version_updates = optional(bool, true)<br/> start = optional(string, "01:00:00Z")<br/> end = optional(string, "02:00:00Z")<br/> }), {})<br/> })<br/> }))</pre> | `{}` | no |
| <a name="input_platform_kubernetes"></a> [platform\_kubernetes](#input\_platform\_kubernetes) | Map of central, region-scoped platform Kubernetes deployments. Empty map skips deployment. | <pre>map(object({<br/> region = string<br/> network = optional(object({<br/> sna_enabled = optional(bool, false)<br/> sna_network_area_id = optional(string, null)<br/> firewall_next_hop_ip = optional(string, null)<br/> sna_network_prefix_length = optional(number, 24)<br/> }), {})<br/> dns = optional(object({<br/> enabled = optional(bool, true)<br/> create_zones = optional(bool, true)<br/> zones = optional(list(string), [])<br/> gateway_api = optional(bool, true)<br/> }), {})<br/> observability = optional(object({<br/> enabled = optional(bool, true)<br/> plan_name = optional(string, "Observability-Starter-EU01")<br/> acl = optional(list(string), [])<br/> name = optional(string, null)<br/> }), {})<br/> encrypted_volumes = optional(object({<br/> enabled = optional(bool, false)<br/> storage_class_name = optional(string, "stackit-encrypted-premium")<br/> kms_keyring_name = optional(string, "ske-volume-keyring")<br/> kms_key_name = optional(string, "ske-volume-key")<br/> kms_key_version = optional(string, "1")<br/> }), {})<br/> debug_bastion = optional(object({<br/> enabled = optional(bool, false)<br/> name = optional(string, null)<br/> availability_zone = optional(string, null)<br/> machine_type = optional(string, "g2i.1")<br/> image_id = optional(string, "7b10e105-295b-4369-b6e0-567ec940a02b")<br/> boot_volume_size = optional(number, 20)<br/> ssh_public_key = optional(string, null)<br/> ssh_public_key_path = optional(string, "~/.ssh/id_rsa.pub")<br/> ssh_allowed_cidrs = optional(list(string), ["0.0.0.0/0"])<br/> assign_public_ip = optional(bool, true)<br/> install_kubectl = optional(bool, true)<br/> }), {})<br/> role_assignments = optional(list(object({<br/> role = string<br/> subject = string<br/> })), [])<br/> cluster = object({<br/> name = string<br/> kubernetes_version_min = optional(string, null)<br/> node_pools = optional(list(object({<br/> name = string<br/> machine_type = string<br/> minimum = number<br/> maximum = number<br/> availability_zones = list(string)<br/> allow_system_components = optional(bool, false)<br/> volume_size = optional(number, 20)<br/> volume_type = optional(string, "storage_premium_perf1")<br/> os_name = optional(string, "flatcar")<br/> labels = optional(map(string), {})<br/> })), [<br/> {<br/> name = "system"<br/> machine_type = "g3i.4"<br/> minimum = 2<br/> maximum = 2<br/> availability_zones = ["eu01-1"]<br/> allow_system_components = true<br/> volume_size = 20<br/> volume_type = "storage_premium_perf1"<br/> os_name = "flatcar"<br/> labels = {<br/> "workload-role" = "system"<br/> }<br/> },<br/> {<br/> name = "application"<br/> machine_type = "g3i.4"<br/> minimum = 2<br/> maximum = 2<br/> availability_zones = ["eu01-2"]<br/> allow_system_components = false<br/> volume_size = 20<br/> volume_type = "storage_premium_perf1"<br/> os_name = "flatcar"<br/> labels = {<br/> "workload-role" = "application"<br/> }<br/> }<br/> ])<br/> maintenance = optional(object({<br/> enable_kubernetes_version_updates = optional(bool, true)<br/> enable_machine_image_version_updates = optional(bool, true)<br/> start = optional(string, "01:00:00Z")<br/> end = optional(string, "02:00:00Z")<br/> }), {})<br/> })<br/> }))</pre> | `{}` | no |
| <a name="input_platform_kubernetes_kube_config_override"></a> [platform\_kubernetes\_kube\_config\_override](#input\_platform\_kubernetes\_kube\_config\_override) | Optional raw kubeconfig used by kubernetes/helm platform providers when no platform\_kubernetes module output is available. | `string` | `null` | no |
| <a name="input_region"></a> [region](#input\_region) | STACKIT region for regional resources. | `string` | `"eu01"` | no |
| <a name="input_rm_folder_parent_id"></a> [rm\_folder\_parent\_id](#input\_rm\_folder\_parent\_id) | ID of the parent folder under which the resource manager folders will be created. If not provided, the resource manager folders will be created under the organization. | `string` | `null` | no |
Expand Down
62 changes: 9 additions & 53 deletions src/_landing-zone-kubernetes.tf
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,15 @@
## LANDING ZONE ON KUBERNETES ##
################################

# The Kubernetes side of a landing zone: every entry in landing_zone_namespace_services
# gets a tenant slice of the shared platform cluster — its own namespace, a scoped service
# account with a Role limited to that namespace, and optionally a Kyverno policy that
# blocks direct Secret management so credentials have to come through the Secrets Manager.
#
# The sample workload behind sample_load is demo material, not part of the landing zone
# contract: a pod and a Gateway API route that together prove the path from the internet
# to a namespace works. Drop it once real workloads move in.

locals {
secrets_enforcement_default_exempt_principals = [
"system:serviceaccount:external-secrets:external-secrets",
Expand Down Expand Up @@ -78,13 +87,6 @@ locals {
for key, value in local.landing_zone_namespace_services : key => value
if value.secrets_enforcement.enabled
}

sample_gateway_lb_endpoint_by_key = {
for key, data in data.kubernetes_resources.landing_zone_sample_gateway_service : key => {
ip = try(one(data.objects).status.loadBalancer.ingress[0].ip, null)
hostname = try(one(data.objects).status.loadBalancer.ingress[0].hostname, null)
}
}
}

module "namespace_service_demo" {
Expand Down Expand Up @@ -592,49 +594,3 @@ resource "kubernetes_manifest" "landing_zone_sample_http_route" {
]
}

data "kubernetes_resources" "landing_zone_sample_gateway_service" {
Comment thread
lweberru marked this conversation as resolved.
provider = kubernetes.platform

for_each = {
for key, value in local.landing_zone_namespace_services : key => value
if value.sample_load.enabled && value.dns_fqdn != null
}

api_version = "v1"
kind = "Service"
namespace = "envoy-gateway-system"
label_selector = "gateway.envoyproxy.io/owning-gateway-name=${kubernetes_manifest.landing_zone_sample_gateway[each.key].manifest.metadata.name},gateway.envoyproxy.io/owning-gateway-namespace=${kubernetes_namespace_v1.landing_zone[each.key].metadata[0].name}"

depends_on = [
kubernetes_manifest.landing_zone_sample_gateway,
]
}

resource "stackit_dns_record_set" "landing_zone_sample_gateway" {
for_each = {
for key, value in local.landing_zone_namespace_services : key => value
if value.sample_load.enabled && value.dns_fqdn != null
}

project_id = module.landing_zone[each.key].project_id
zone_id = module.landing_zone[each.key].dns_zone_id
name = each.value.dns_fqdn
type = try(local.sample_gateway_lb_endpoint_by_key[each.key].ip, null) != null ? "A" : "CNAME"
ttl = 60
records = [
coalesce(
try(local.sample_gateway_lb_endpoint_by_key[each.key].ip, null),
try(local.sample_gateway_lb_endpoint_by_key[each.key].hostname, null),
),
]

lifecycle {
precondition {
condition = (
try(local.sample_gateway_lb_endpoint_by_key[each.key].ip, null) != null ||
try(local.sample_gateway_lb_endpoint_by_key[each.key].hostname, null) != null
)
error_message = "Gateway load balancer endpoint is not available yet for DNS record creation."
}
}
}
1 change: 1 addition & 0 deletions src/main.tf
Original file line number Diff line number Diff line change
Expand Up @@ -131,6 +131,7 @@ module "platform_kubernetes" {
enabled = each.value.dns.enabled
create_zones = each.value.dns.create_zones
zones = length(each.value.dns.zones) > 0 ? each.value.dns.zones : compact(distinct([for lz in values(module.landing_zone) : try(lz.dns_zone_dns_name, null)]))
gateway_api = each.value.dns.gateway_api
}
}

Expand Down
5 changes: 3 additions & 2 deletions src/modules/platform-kubernetes/3-cluster.tf
Original file line number Diff line number Diff line change
Expand Up @@ -36,8 +36,9 @@ resource "stackit_ske_cluster" "this" {
instance_id = var.observability.enabled ? stackit_observability_instance.this[0].instance_id : null
}
dns = {
enabled = var.dns.enabled && length(local.effective_dns_zones) > 0
zones = local.effective_dns_zones
enabled = var.dns.enabled && length(local.effective_dns_zones) > 0
zones = local.effective_dns_zones
gateway_api = var.dns.enabled && length(local.effective_dns_zones) > 0 ? var.dns.gateway_api : null
}
}

Expand Down
Loading
Loading