Skip to content

feat(metrics): report scheduler utilization in runtime metrics - #1189

Open
solnic wants to merge 1 commit into
feat/runtime-metrics-system-limitsfrom
feat/runtime-metrics-scheduler-utilization
Open

solnic wants to merge 1 commit into
feat/runtime-metrics-system-limitsfrom
feat/runtime-metrics-scheduler-utilization

Conversation

@solnic

@solnic solnic commented Sep 3, 2026

Copy link
Copy Markdown
Collaborator

Adds elixir.runtime.scheduler.utilization, a number between 0.0 and 1.0 for how much of the VM's capacity to run code is actually in use. 0.0 is idle, 1.0 means there is no headroom left.

In practical terms, this will show you how busy the VM's main worker threads were - running application code plus the runtime's own work (GC, memory management, drivers, BIFs) - excluding time blocked on I/O.

Screenshot

If this is close to 1.0 very often for significant periods, you may want an alert set up.

image

@solnic
solnic force-pushed the feat/runtime-metrics-scheduler-utilization branch from e952b37 to fabb545 Compare September 4, 2026 12:58
@solnic
solnic force-pushed the feat/runtime-metrics-scheduler-utilization branch from fabb545 to bf492a4 Compare September 4, 2026 13:28
@solnic
solnic force-pushed the feat/runtime-metrics-scheduler-utilization branch from bf492a4 to 4cd789b Compare September 9, 2026 13:13
@solnic
solnic force-pushed the feat/runtime-metrics-scheduler-utilization branch from 4cd789b to c77b098 Compare September 9, 2026 14:34
@solnic
solnic force-pushed the feat/runtime-metrics-scheduler-utilization branch 2 times, most recently from fda8ed5 to e388efc Compare September 11, 2026 09:52
@solnic
solnic removed this pull request from stack #1192 September 11, 2026 09:57
@solnic
solnic added this pull request to stack #1206 September 11, 2026 09:58
@solnic
solnic force-pushed the feat/runtime-metrics-scheduler-utilization branch from e388efc to 6fd4c6b Compare September 11, 2026 12:18
@solnic
solnic force-pushed the feat/runtime-metrics-scheduler-utilization branch from 6fd4c6b to e08be02 Compare September 11, 2026 12:35
@solnic
solnic removed this pull request from stack #1206 September 14, 2026 08:49
@solnic
solnic added this pull request to stack #1210 September 15, 2026 08:50
@solnic
solnic force-pushed the feat/runtime-metrics-scheduler-utilization branch from e08be02 to 76795fa Compare September 15, 2026 09:13
@solnic
solnic force-pushed the feat/runtime-metrics-scheduler-utilization branch from 76795fa to a89784a Compare September 15, 2026 13:44
@solnic
solnic force-pushed the feat/runtime-metrics-scheduler-utilization branch from a89784a to 6a8288a Compare September 15, 2026 14:28
@solnic
solnic marked this pull request as ready for review September 16, 2026 08:20
@solnic
solnic force-pushed the feat/runtime-metrics-scheduler-utilization branch from 6a8288a to 3515519 Compare September 16, 2026 13:27
@solnic
solnic removed this pull request from stack #1210 September 16, 2026 13:27
@solnic
solnic added this pull request to stack #1211 September 16, 2026 13:28
@solnic
solnic removed this pull request from stack #1211 September 16, 2026 13:33
@solnic
solnic changed the base branch from feat/runtime-metrics-collector to feat/runtime-metrics-system-limits September 16, 2026 13:34
@solnic
solnic added this pull request to stack #1214 September 16, 2026 13:34
@solnic
solnic marked this pull request as draft September 16, 2026 13:37
@solnic
solnic force-pushed the feat/runtime-metrics-scheduler-utilization branch from 3515519 to 7cbd054 Compare September 16, 2026 13:55
@solnic
solnic force-pushed the feat/runtime-metrics-scheduler-utilization branch 4 times, most recently from 6b3eed1 to 45f4265 Compare September 17, 2026 08:32
@solnic
solnic force-pushed the feat/runtime-metrics-scheduler-utilization branch 2 times, most recently from 1dbe3ae to ec0c38b Compare September 17, 2026 10:00
@solnic
solnic force-pushed the feat/runtime-metrics-scheduler-utilization branch 2 times, most recently from 2839899 to 29b6766 Compare September 17, 2026 11:27
@solnic
solnic force-pushed the feat/runtime-metrics-scheduler-utilization branch from 29b6766 to 6eef7d2 Compare September 17, 2026 11:54
@solnic
solnic force-pushed the feat/runtime-metrics-scheduler-utilization branch from 6eef7d2 to e2616d8 Compare September 17, 2026 13:14
@solnic
solnic marked this pull request as ready for review September 17, 2026 13:15
@solnic
solnic force-pushed the feat/runtime-metrics-scheduler-utilization branch from e2616d8 to 061c090 Compare September 18, 2026 07:16
Comment on lines +113 to +122
defp dispatch_utilization(previous, current) do
{active, total} =
previous
|> Enum.zip(current)
|> Enum.reduce({0, 0}, fn {{_, active0, total0}, {_, active1, total1}}, {active, total} ->
{active + (active1 - active0), total + (total1 - total0)}
end)

:telemetry.execute(@scheduler_event, %{utilization: ratio(active, total)}, %{})
end

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: The dispatch_utilization/2 function is missing a guard for an empty previous argument, which can cause a spurious 0.0 metric to be emitted on the first successful sample.
Severity: LOW

Suggested Fix

Add a function clause to dispatch_utilization/2 to handle cases where the first argument is an empty list, preventing the incorrect metric calculation. For example: defp dispatch_utilization([], _current), do: :ok.

Prompt for AI Agent
Review the code at the location below. A potential bug has been identified by an AI
agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's
not valid.

Location: lib/sentry/metrics/runtime.ex#L113-L122

Potential issue: The `dispatch_utilization/2` function does not handle cases where the
`previous` sample is an empty list but the `current` sample is not. This can occur if
the first call to `scheduler_sample()` returns `[]` (due to
`:erlang.statistics(:scheduler_wall_time)` returning `:undefined` initially) and a
subsequent call returns valid data. In this scenario, `Enum.zip([], current_data)`
results in an empty list, the reduce operation defaults to `{0, 0}`, and `ratio(0, 0)`
calculates `0.0`. This leads to a spurious `0.0` utilization metric being emitted for a
single polling cycle before normal operation resumes.

Also affects:

  • lib/sentry/metrics/runtime.ex:96~102

Did we get this right? 👍 / 👎 to inform future reviews.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant