design-proposal: public IPs as a first-class resource (PublicIP / PublicIPClaim / PublicIPClass)

[Timofei Larkin (lllamnyp)]

What

Adds a design proposal under design-proposals/public-ip-claims/ introducing public addresses as a first-class resource: a cluster-scoped PublicIP, a namespaced PublicIPClaim, a PublicIPClass, and a provisioner contract in the shape of CSI.

Status: stub. The problem statement, the primitives survey and the object model are worked out. The controller mechanics, the provisioner contract details and the rollout are deliberately sketched. It is opened early to settle the shape — is an address a resource? — before anyone writes code.

Why

A tenant cannot own a public address. One appears as a side effect of creating a Service type: LoadBalancer, and evaporates when that Service is deleted. So an address cannot be:

• held across a workload rebuild (delete the Service, lose the address — someone else may get it),
• moved from one workload to another during a cutover,
• quota'd (there is no object to count; the only lever is pre-carving a pool per tenant),
• gated (anyone who can create a Service can cause an address to be allocated — RBAC authorizes verbs on resources, not fields),
• enumerated ("which addresses do we own, and who holds them?" has no answer short of listing every Service in the cluster).

The experience we want is the AWS one: allocate an address, keep it, attach it, detach it, attach it to something else, release it when done.

This is a regression, not a wishlist

Cozystack has reserve-then-associate today, via kube-ovn: OvnEip is created independently and later bound by an OvnFip / OvnSnatRule / OvnDnatRule that references it by name. Migrating off kube-ovn drops that capability unless something replaces it. The question is not "should Cozystack gain an AWS-style EIP" but "on what terms does it keep the one it already has."

The model

The PersistentVolume pattern, applied to addresses:

storage	addresses	role
StorageClass	PublicIPClass	which pool, which provisioner, which announcer
PersistentVolume	PublicIP (cluster-scoped)	the address, with a claimRef and a reclaim policy
PersistentVolumeClaim	PublicIPClaim (namespaced)	"give me one" — the whole tenant-facing API
CSI driver	a provisioner contract	how a class's backend allocates, adopts, and pins

Two things fall out of the survey and are worth a reviewer's attention:

• An address source union (fromClass vs providerRef), directly analogous to PV's volume-source union. Self-allocating backends (MetalLB, Cilium, kube-vip) have no reservation concept anywhere, so our provisioner is the IPAM of record. Cloud backends (AWS, GCP) already hold the reservation with a stable handle and their own IAM, so the provisioner must adopt, not allocate. Surviving a backend where allocation isn't ours is the evidence the shape isn't bent around MetalLB.
• A Pin capability, which the entire model depends on: a backend that cannot be told which address to use can never attach a reserved one, so a class over it must reject claims outright rather than allocate an address it can never bind.

The sharp edge (please read the Security section)

The per-backend "pin this IP" annotation (metallb.io/loadBalancerIPs, lbipam.cilium.io/ips, …) is a privilege-escalation surface. A self-allocating backend has no concept of a reservation, so it will hand a tenant an address that another tenant has reserved but not yet attached. That theft window exists on MetalLB and Cilium alike.

The fix is the move Cozystack already makes elsewhere: turn the ungatable field into a reference to an RBAC-gated object. Tenants may only write network.cozystack.io/public-ip-claim (naming an object in their own namespace); the controller writes the raw pin annotation; a ValidatingAdmissionPolicy rejects any other principal writing one. Without that policy the model is advisory, so it belongs in the same release as the CRDs.

Notes for reviewers

• Adjacent to, but not dependent on, design-proposal: structured, additive external exposure for managed applications #29 (structured-external-exposure). That proposal restructures how an application requests exposure; this one is about what an address is. They meet at one point — an exposure entry ought to be able to name a claim instead of implicitly minting an address — but neither needs the other to land first, and this does not assume design-proposal: structured, additive external exposure for managed applications #29's shape.
• ExposureClass is touched on but not depended on. The class idea is right and survives here in some form; binding an address's lifetime to a Service is the part that structurally cannot serve this use case. Open question 4 defers the merge-or-separate call, since that kind's future is being re-examined independently.
• The datapath is out of scope. Whole-IP 1:1 NAT (all ports in, and the workload egressing as that address — today cozy-proxy's nftables rules) is a CNI concern tracked separately. This is the allocation half; that is the forwarding half. Allocation is useful on its own with a stock LoadBalancer Service.
• Nothing upstream is coming. KEP-1880 shipped ServiceCIDR/IPAddress and explicitly named "any generalization onto something like an IPAM API" a non-goal; loadBalancerIP was deprecated in 1.24 with no core replacement. Cluster API's IPAddressClaim/IPAddress is literally this pattern but has only ever been wired to Machine addressing — open question 3 asks whether we adopt it or merely copy the contract.

The test that is the proposal

Claim → bind → associate → delete the Service → the address is still held → associate it to a different workload → the same address comes back.

That single integration test is precisely what cannot pass today.

Summary by CodeRabbit

• Documentation
  • Added a draft proposal for reservable IP address resources in Kubernetes.
  • Described new tenant and administrator workflows for requesting, allocating, adopting, and reclaiming IP addresses.
  • Documented proposed resource types, provisioning capabilities, quota integration, security safeguards, migration guidance, failure handling, and rollout considerations.
  • Included support considerations for dual-stack addresses, provider-managed reservations, sharing, and eventual consistency.