Skip to content

docs: stable nodepool tolerations and observability resource profiles #191

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Guimove wants to merge 3 commits into
base: main
Choose a base branch
from
+49 −0
Open

docs: stable nodepool tolerations and observability resource profiles #191

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
c516bac
docs: document stable nodepool tolerations and observability resource…
Guimove Sep 17, 2026
98e2f7d
docs: address review feedback on nodepool tolerations and profiles
Guimove Sep 17, 2026
9176414
docs: scope the resource profile to the metrics components
Guimove Sep 17, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 31 additions & 0 deletions docs/configuration/integrations/observability/qovery-observe.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,37 @@ Qovery's observability combines open-source tools to monitor your Kubernetes inf
- **Thanos**: Raw metrics (15 days), 5-minute resolution (30 days), 1-hour resolution (30 days)
- **Loki**: 12-week log retention

### Resource Profiles

The observability stack runs under one of three resource profiles, set on the cluster through the `metrics_parameters.configuration.resource_profile` API field. `NORMAL` is the default.

The profile changes the CPU and memory requests and limits of the metrics components listed below, and nothing else. Retention, scrape interval, replica counts and alerting configuration are the same on all three levels. Log and event collection are not affected: Loki, Promtail and the Qovery Event Logger are sized independently of this setting.

CPU is shown as request/limit. Memory is shown as a single value when the request and the limit are equal, and as request/limit when they differ.

| Component | LOW | NORMAL | HIGH |
|---|---|---|---|
| Prometheus | 500m/2000m, 1Gi | 1000m/4000m, 4Gi | 2000m/4000m, 8Gi |
| Prometheus Operator | 200m/1000m, 512Mi/1Gi | 200m/1000m, 1Gi | 500m/1000m, 1Gi |
| Node Exporter | 10m/20m, 32Mi | 10m/20m, 32Mi | 10m/20m, 32Mi |
| kube-state-metrics | 150m/150m, 512Mi | 150m/150m, 768Mi | 500m/500m, 1Gi |
| Thanos Query | 500m/500m, 512Mi | 1000m/1000m, 768Mi | 2000m/2000m, 1Gi |
| Thanos Store Gateway | 500m/500m, 512Mi | 500m/500m, 1Gi | 1000m/1000m, 2Gi |
| Thanos Compactor | 500m/500m, 1Gi | 2000m/2000m, 4Gi | 2000m/2000m, 6Gi |
| Prometheus Adapter | 250m/250m, 384Mi | 250m/250m, 384Mi | 400m/400m, 512Mi |
| AlertManager | 50m/100m, 128Mi/256Mi | 100m/200m, 256Mi/512Mi | 200m/500m, 512Mi/1Gi |
| CloudWatch Exporter | 150m/150m, 256Mi | 250m/250m, 512Mi | 500m/500m, 768Mi |

<Info>
Lowering the profile reduces what the observability stack reserves on your cluster, not the data it keeps. Retention stays as described in [Data Retention](#data-retention) on every profile.
</Info>

<Warning>
`LOW` is intended for small or development clusters. It cuts the memory available to Prometheus and to the Thanos Compactor, as shown in the LOW column above. Compaction is memory-bound, so on clusters with a high number of series those limits can cause OOMKills.
</Warning>

A profile change applies on the next cluster deployment.

### Key Features

- **Per-cluster isolation**: Data protection and performance optimization
Expand Down
18 changes: 18 additions & 0 deletions docs/configuration/service-advanced-settings.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -191,6 +191,24 @@ This value must be **greater than** the time your `deployment.lifecycle.pre_stop
If no node matches the required labels, your pods will stay in `Pending` state. Make sure the target node pool or nodes exist before applying this setting.
</Warning>

#### Targeting the `stable` node pool on AWS with Karpenter

The `stable` node pool carries a `nodepool/stable:NoSchedule` taint. Node affinity alone is not enough: a pod also needs a matching toleration, and this setting only controls the affinity.

Qovery adds the toleration for you on **Applications** and **Containers**. Setting `{"karpenter.sh/nodepool": "stable"}` on those services is enough, and their pods schedule normally. Container-mode databases are pinned to `stable` automatically, with their toleration, and need no setting at all.

<Warning>
On **Jobs** and **Cronjobs**, the `stable` toleration is not added. Setting `{"karpenter.sh/nodepool": "stable"}` on those services applies the affinity without the toleration, and their pods stay in `Pending`.
</Warning>

This is separate from the optional `cronjob` node pool, which has its own taint and its own automatic toleration. See [Dedicated Cronjob Node Pool](/configuration/cronjob#dedicated-cronjob-node-pool).

<Info>
On EKS clusters with Karpenter, Applications and Containers are placed on the `stable` node pool automatically, affinity and toleration included, when the service has `min_instances` set to `1` or when it has storage attached. In those cases you do not need to set this advanced setting yourself.
</Info>

For Helm services, Qovery does not inject affinity or tolerations, so you declare both in your own `values.yaml`. See [Change the node pool of your service when using Helm](/configuration/integrations/kubernetes/eks/managed#change-the-node-pool-of-your-service-when-using-helm).

<a id="deployment-antiaffinity-pod"></a>

### deployment.antiaffinity.pod
Expand Down
Loading