From 32a041e949d56384a3ff59b15bccc78680510cac Mon Sep 17 00:00:00 2001 From: krisztianfekete Date: Mon, 21 Sep 2026 12:25:50 +0200 Subject: [PATCH] fix(helm): let the chart turn on actor lifecycle events --- charts/substrate/README.md | 4 ++-- charts/substrate/templates/_helpers.tpl | 8 ++++++++ charts/substrate/values.yaml | 3 ++- docs/dev/best-practices/otel-collector.md | 3 ++- docs/observability.md | 4 ++-- 5 files changed, 16 insertions(+), 6 deletions(-) diff --git a/charts/substrate/README.md b/charts/substrate/README.md index 2803a5aa63..7fd7b09ed1 100644 --- a/charts/substrate/README.md +++ b/charts/substrate/README.md @@ -49,11 +49,11 @@ See `values.yaml` for the full set; the important keys: | `atelet.gcpAuthForImagePulls` | `false` | Enable only when using GCP registry auth | | `credentialProvider.namespacePolicies` | `[]` | Default-deny atespace-to-namespace grants; the chart includes get-only Secret RBAC for the provider | | `ateApi.extraArgs` | `[]` | Additional command-line arguments appended to the ateapi defaults | -| `otel.endpoint` | `""` | Set to an OTLP endpoint to export traces, metrics and the router access log | +| `otel.endpoint` | `""` | Set to an OTLP endpoint to export traces, metrics, the actor lifecycle events and the router access log | | `otel.traces.enabled` | `true` | Set to `false` to export no traces from the router; the Go components do not honor this yet | | `otel.traces.endpoint` | `""` | OTLP endpoint for traces, overriding `otel.endpoint` | | `otel.traces.samplingRatio` | `0.01` | Fraction of parentless requests that start a trace, applied to the Go components and the router | | `otel.metrics.enabled` | `true` | Sets the OTLP metrics exporter to `none`; the Go components do not honor this yet | | `otel.metrics.endpoint` | `""` | OTLP endpoint for metrics, overriding `otel.endpoint` | -| `otel.logs.enabled` | `true` | Set to `false` to export no logs; the router access log is the only OTLP log source today | +| `otel.logs.enabled` | `true` | Set to `false` to export no logs. Gates both OTLP log sources: ateapi's actor lifecycle events and the router access log | | `otel.logs.endpoint` | `""` | OTLP endpoint for logs, overriding `otel.endpoint` | diff --git a/charts/substrate/templates/_helpers.tpl b/charts/substrate/templates/_helpers.tpl index 3b81dc9c0b..45184413fb 100644 --- a/charts/substrate/templates/_helpers.tpl +++ b/charts/substrate/templates/_helpers.tpl @@ -124,6 +124,14 @@ Usage: value: {{ $cfg.endpoint | quote }} {{- end }} {{- end }} +{{- if include "substrate.otel.signalEndpoint" (list "logs" .) }} +{{- /* Only logs need turning on: serverboot defaults the component to none, so + an enabled signal exports nothing without this. Traces and metrics always + export, so they need no such branch -- keep this out of the range above. + Gated on an endpoint, since otlp without one retries localhost:4317. */}} +- name: OTEL_LOGS_EXPORTER + value: otlp +{{- end }} {{- if include "substrate.otel.signalEndpoint" (list "traces" .) }} - name: OTEL_TRACES_SAMPLER value: parentbased_traceidratio diff --git a/charts/substrate/values.yaml b/charts/substrate/values.yaml index fef293a355..618be970b1 100644 --- a/charts/substrate/values.yaml +++ b/charts/substrate/values.yaml @@ -77,7 +77,8 @@ ateApiServerEnvVarsConfigMap: ate-api-server-envvars # traces.enabled and metrics.enabled set OTEL__EXPORTER=none on the # Go components, which do not read it yet and keep exporting; the setting # takes effect on the router's agentgateway only. logs.enabled works in full, -# since the agentgateway access log is the only OTLP log source. +# and gates both OTLP log sources: ateapi's actor lifecycle events and the +# router's agentgateway access log. otel: endpoint: "" traces: diff --git a/docs/dev/best-practices/otel-collector.md b/docs/dev/best-practices/otel-collector.md index 996f4f1adb..ef072c9916 100644 --- a/docs/dev/best-practices/otel-collector.md +++ b/docs/dev/best-practices/otel-collector.md @@ -435,7 +435,8 @@ for a worked example. **Substrate exports one thing over OTLP: the actor lifecycle events**, from ateapi, through `serverboot.InitLogging`. They are off unless -`OTEL_LOGS_EXPORTER=otlp` is set, which only the kind overlay does today. See +`OTEL_LOGS_EXPORTER=otlp` is set. The kind overlay sets it, and the Helm chart +sets it from `otel.logs.enabled` once `otel.endpoint` resolves. See [the same records over OTLP](../../observability.md#the-same-records-over-otlp). Everything else is stdout. `serverboot.InitLogger` writes structured JSON there, diff --git a/docs/observability.md b/docs/observability.md index 2ddc37be49..b8c512cd04 100644 --- a/docs/observability.md +++ b/docs/observability.md @@ -178,7 +178,7 @@ The counter carries the same reason but no actor identity, so this record is the #### The same records over OTLP -Both records also go out as OTLP log events, so a collector reads them without knowing substrate's stdout envelope. Set `OTEL_LOGS_EXPORTER=otlp` to turn it on; unset means `none`, which is what every environment but kind uses today. Only ateapi has a LoggerProvider — a worker pod cannot export a log record at all yet, because [the ateom relay](#the-ateom-otlp-relay) carries traces and metrics only. +Both records also go out as OTLP log events, so a collector reads them without knowing substrate's stdout envelope. Set `OTEL_LOGS_EXPORTER=otlp` to turn it on; unset means `none`. The kind overlay sets it, and a chart install gets it from `otel.logs.enabled` once `otel.endpoint` resolves. Only ateapi has a LoggerProvider — a worker pod cannot export a log record at all yet, because [the ateom relay](#the-ateom-otlp-relay) carries traces and metrics only. Two `event.name` values, which is the OTLP LogRecord's own field rather than an attribute: @@ -405,7 +405,7 @@ Telemetry is emitted the same way everywhere; only the backend differs between a | Path | service → in-cluster `opentelemetry-collector` | service → Google Managed Prometheus (GMP) | | Metrics | collector Prometheus exporter on `:8889` | Google Cloud Monitoring | | Traces | Jaeger UI | Google Cloud Trace | -| Logs | pod stdout; ateapi's [actor lifecycle events](#the-same-records-over-otlp) also to the collector's `debug` exporter | pod stdout. No OTLP logs: `OTEL_LOGS_EXPORTER` is unset | +| Logs | pod stdout; ateapi's [actor lifecycle events](#the-same-records-over-otlp) also to the collector's `debug` exporter | pod stdout. No OTLP logs unless `OTEL_LOGS_EXPORTER` is set | | Dashboards | Not supported | Google Cloud Monitoring (see [Dashboards](#5-dashboards)) | > In Kind, `ateapi`, `atelet`, `ate-controller`, and `atenet-router` are pointed at the in-cluster collector, and the controller propagates the endpoint to the ateom worker pods it creates, so all component telemetry lands locally.