PROF-16213: Account CTimer table in NativeMem and gate thread registration

AGENTS.md

@@ -146,7 +146,7 @@ before working on a subsystem.
 | String dictionary (arena chunks, rotation/clear protocols) | [doc/architecture/StringDictionary.md](doc/architecture/StringDictionary.md) |
 | RefCountGuard (lock-free RAII reader-drain before reclamation) | [doc/architecture/RefCountGuard.md](doc/architecture/RefCountGuard.md) |
 | TLS context (thread-local tracing context, distributed tracing) | [doc/architecture/TLSContext.md](doc/architecture/TLSContext.md) |
-| Profiling engines (CPU, wall-clock, allocation, native memory, liveness) | [doc/architecture/ProfilingEngines.md](doc/architecture/ProfilingEngines.md) |
+| Profiling engines (CPU, wall-clock, allocation, native memory, liveness; thread-registration gates) | [doc/architecture/ProfilingEngines.md](doc/architecture/ProfilingEngines.md) |
 | JFR integration (FlightRecorder, metadata, constant pools, buffers) | [doc/architecture/JfrIntegration.md](doc/architecture/JfrIntegration.md) |
 | Live-heap reference chains (implementation, signal scheduling, design) | [doc/architecture/LiveHeapReferenceChainsImplementation.md](doc/architecture/LiveHeapReferenceChainsImplementation.md), [doc/architecture/ReferenceChainsSignalsExplained.md](doc/architecture/ReferenceChainsSignalsExplained.md), [doc/ReferenceChainsDesign.md](doc/ReferenceChainsDesign.md), [doc/ReferenceChainsCollectionSummary.md](doc/ReferenceChainsCollectionSummary.md) |
 | Native memory profiling (malloc/free interception, sampling) | [doc/architecture/NativeMemoryProfiling.md](doc/architecture/NativeMemoryProfiling.md) |
@@ -272,6 +272,42 @@ Otherwise a signal can sample a half-torn-down thread, or re-acquire a
 thread that keeps running silently stops profiling that thread — use `SignalBlocker`
 for a temporary window.
 
+**Thread registration.** The `pthread_create` hook and the JVMTI `ThreadStart`/
+`ThreadEnd` callbacks run without `_state_lock` and can overlap `Profiler::start()`/
+`stop()`, while engines size, replace or tear down their per-thread tables (`CTimer`'s
+`_timers`, `PerfEvents`' `_events`). So:
+- reach the engines only through `Profiler::registerThread()`/`unregisterThread()`,
+  never `_cpu_engine->registerThread()` directly. These calls check a gate
+  (`_engines_ready` / `_thread_tables_stable`), count themselves in flight
+  (`HookCallScope`), and run with thread cancellation disabled;
+- an engine with a per-thread table sizes it first, then calls
+  `Profiler::openThreadRegistration()`, then lists the existing threads. Free or
+  replace a table only while its gate is closed: closing a gate
+  (`closeThreadRegistration()`, `ThreadTablesFreeze`) waits until no call is inside an
+  engine. Each gate counts its calls separately: `closeThreadRegistration()` waits only
+  for registration calls, because unregistration stays open during `stop()`;
+  `ThreadTablesFreeze` waits for both. The wait is bounded; when it reports failure,
+  free or replace no table. A `ThreadTablesFreeze` timeout fails the operation; a
+  `closeThreadRegistration()` timeout puts the profiler in `ERROR` (`stop()` still
+  completes, since it frees no table), so no later `start()` replaces a table under
+  the stuck call. Engine
+  resources kept across `stop()` are released (`Engine::releaseThreads()`) when another
+  engine replaces the CPU engine. Known exception: `PerfEvents::start()` still
+  replaces `_events` without a gate (PROF-16359). `Profiler::startCpuEngine()` opens the
+  registration gate once any CPU engine started, so the gate is open iff it did, and it
+  admits only the CPU engine. A wall engine that needs `registerThread()` needs its own
+  gate: opening this one would also admit a CPU engine that failed;
+- an engine's `stop()` must tolerate a concurrent `unregisterThread()`: `stop()` closes
+  only the registration gate, and unregistration stays open while the engines stop;
+- close unregistration (`ThreadTablesFreeze`) only around an actual table replacement
+  or a CPU engine change, never for a whole `start()`: an exit dropped while it is closed leaves a stale slot
+  that a thread reusing the tid cannot claim, and `start()`/`stop()` cycles repeat;
+- restore the cancellation state outside every destructor-bearing scope, never from a
+  destructor: current glibc acts on a pending asynchronous cancellation as soon as it
+  is re-enabled, and unwinding out of an implicitly `noexcept` destructor terminates.
+
+Details: [ProfilingEngines.md](doc/architecture/ProfilingEngines.md#thread-registration).
+
 **JVMTI callbacks.** Callbacks run on whichever JVM thread triggers the event —
 including compiler, VM and GC threads that never went through our `pthread_create`
 hook — so call `ProfiledThread::initCurrentThreadSignalSafe()` first.

ddprof-lib/src/main/cpp/ctimer.h

@@ -1,5 +1,6 @@
 /*
  * Copyright 2017 Andrei Pangin
+ * Copyright 2026, Datadog, Inc.
  *
  * Licensed under the Apache License, Version 2.0 (the "License");
  * you may not use this file except in compliance with the License.
@@ -25,6 +26,8 @@
 #include <signal.h>
 
 class CTimer : public Engine {
+  friend class CTimerTestAccessor;
+
 private:
   // cppcheck-suppress unusedPrivateFunction
   static void signalHandler(int signo, siginfo_t *siginfo, void *ucontext);
@@ -41,6 +44,13 @@ class CTimer : public Engine {
   static int _max_timers;
   static int *_timers;
 
+  static bool resizeTimers(int max_timers);
+  // Sizes the timer table for start(), replacing it under a
+  // Profiler::ThreadTablesFreeze only when the size changes.
+  static Error sizeTimerTable(int max_timers);
+  // start() with the timer table size passed in rather than read from
+  // pid_max, so a test can drive the allocation-failure path.
+  Error startWithTableSize(Arguments &args, int max_timers);
   int registerThread(int tid);
   void unregisterThread(int tid);
 

ddprof-lib/src/main/cpp/ctimer_linux.cpp

@@ -24,6 +24,7 @@
 #include "debugSupport.h"
 #include "libraries.h"
 #include "log.h"
+#include "nativeMem.h"
 #include "profiler.h"
 #include "samplerPerf.h"
 #include "signalCookie.h"
@@ -54,7 +55,7 @@ bool CTimer::_enabled = false;
 int CTimer::_signal;
 
 int CTimer::registerThread(int tid) {
-  if (tid >= _max_timers) {
+  if (tid < 0 || tid >= _max_timers) {
     Log::warn("tid[%d] > pid_max[%d]. Restart profiler after changing pid_max",
               tid, _max_timers);
     return -1;
@@ -119,7 +120,7 @@ int CTimer::registerThread(int tid) {
 }
 
 void CTimer::unregisterThread(int tid) {
-  if (tid >= _max_timers) {
+  if (tid < 0 || tid >= _max_timers) {
     return;
   }
   // Atomic acquire to avoid possible leak when unregistering
@@ -131,6 +132,51 @@ void CTimer::unregisterThread(int tid) {
   }
 }
 
+// Sizes the tid-indexed timer table and accounts it under NM_CTIMER. Called
+// from sizeTimerTable() under a Profiler::ThreadTablesFreeze, so no
+// thread-hook call (pthread hook or JVMTI ThreadStart/ThreadEnd) can be inside
+// the table while it is replaced.
+//
+// On allocation failure the table is left empty (_max_timers == 0), so the tid
+// bound checks reject every tid instead of indexing a NULL table, and the next
+// start() retries. The table currently held always has _max_timers slots, so
+// that is the size released from NM_CTIMER.
+bool CTimer::resizeTimers(int max_timers) {
+  if (max_timers == _max_timers) {
+    return true;
+  }
+  free(_timers);
+  NativeMem::record(NM_CTIMER, -(long long)((size_t)_max_timers * sizeof(int)));
+  _timers = (int *)calloc(max_timers, sizeof(int));
+  if (_timers == NULL) {
+    _max_timers = 0;
+    return false;
+  }
+  _max_timers = max_timers;
+  NativeMem::record(NM_CTIMER, (long long)((size_t)max_timers * sizeof(int)));
+  return true;
+}
+
+// A restart with an unchanged pid_max keeps the table and never closes thread
+// unregistration, so an exit during start() still releases its timer. Only an
+// actual replacement freezes the hook calls. If one is still inside the table
+// after the wait budget, start() fails and keeps the old table; whether a
+// later start() can succeed depends on the stuck call (see
+// Profiler::startCpuEngine()).
+Error CTimer::sizeTimerTable(int max_timers) {
+  if (max_timers == _max_timers) {
+    return Error::OK;
+  }
+  Profiler::ThreadTablesFreeze freeze(Profiler::instance());
+  if (!freeze.drained()) {
+    return Error("thread hook calls did not drain");
+  }
+  if (!resizeTimers(max_timers)) {
+    return Error("Failed to allocate CPU timer table");
+  }
+  return Error::OK;
+}
+
 Error CTimer::check(Arguments &args) {
   timer_t timer;
   if (timer_create(CLOCK_THREAD_CPUTIME_ID, NULL, &timer) < 0) {
@@ -142,6 +188,10 @@ Error CTimer::check(Arguments &args) {
 }
 
 Error CTimer::start(Arguments &args) {
+  return startWithTableSize(args, OS::getMaxThreadId());
+}
+
+Error CTimer::startWithTableSize(Arguments &args, int max_timers) {
   if (args._interval < 0) {
     return Error("interval must be positive");
   }
@@ -150,11 +200,9 @@ Error CTimer::start(Arguments &args) {
   _cstack = args._cstack;
   _signal = SIGPROF;
 
-  int max_timers = OS::getMaxThreadId();
-  if (max_timers != _max_timers) {
-    free(_timers);
-    _timers = (int *)calloc(max_timers, sizeof(int));
-    _max_timers = max_timers;
+  Error error = sizeTimerTable(max_timers);
+  if (error) {
+    return error;
   }
 
   // Prime the origin-check cache from this non-signal context before any
@@ -165,6 +213,10 @@ Error CTimer::start(Arguments &args) {
 
   OS::installSignalHandler(_signal, signalHandler);
 
+  // The table is sized: open the hook gate before listing threads (see
+  // Profiler::openThreadRegistration()).
+  Profiler::instance()->openThreadRegistration();
+
   // Register all existing threads. Individual failures are benign — a thread
   // may exit between listThreads() and registerThread(), and new threads
   // will register themselves on creation. check() already validated that the

ddprof-lib/src/main/cpp/engine.h

@@ -1,5 +1,6 @@
 /*
  * Copyright 2017 Andrei Pangin
+ * Copyright 2026, Datadog, Inc.
  *
  * Licensed under the Apache License, Version 2.0 (the "License");
  * you may not use this file except in compliance with the License.
@@ -53,6 +54,10 @@ class Engine {
 
   virtual int registerThread(int tid) { return -1; }
   virtual void unregisterThread(int tid) {}
+  // Releases the per-thread resources the engine keeps across stop(). Called
+  // when another engine replaces it in its Profiler slot: thread exits no
+  // longer reach it after that, so nothing else would release them.
+  virtual void releaseThreads() {}
 
   virtual void enableEvents(bool enabled) {
     // do nothing

ddprof-lib/src/main/cpp/nativeMem.h

@@ -14,17 +14,32 @@
 // per-category live byte gauges partition the total: sum(category) == total,
 // with no double counting.
 //
-// These gauges track memory that is actually touched (and therefore resident),
-// not address space that has merely been reserved. NM_CALLTRACE in particular
-// counts the bytes LinearAllocator::alloc() has bump-allocated, NOT the
-// capacity of the 8 MiB chunks backing them -- see linearAllocator.cpp.
+// With the exceptions below, these gauges track memory that is actually touched
+// (and therefore resident), not address space that has merely been reserved.
+// NM_CALLTRACE in particular counts the bytes LinearAllocator::alloc() has
+// bump-allocated, NOT the capacity of the 8 MiB chunks backing them -- see
+// linearAllocator.cpp.
+//
+// Exceptions: the tid-indexed tables of CTimer (NM_CTIMER,
+// pid_max * sizeof(int)) and PerfEvents (part of NM_PERF,
+// pid_max * sizeof(PerfEvent), two cache lines per slot) are counted at their
+// full calloc'd size. That memory is mostly untouched address space -- only the
+// pages holding live tids become resident -- so for these two categories the
+// gauge overstates RSS and should be read as an upper bound. NM_PERF's
+// per-thread ring pages (2 pages each) are touched and resident.
 //
 // Consequently NM_CALLTRACE is no longer an "arena reserved" figure with
 // CALLTRACE_STORAGE_BYTES as its used slice: both now count touched bytes, and
 // their small difference is the call-trace hash tables (also bump-allocated),
 // not chunk slack. Do not compute arena waste as the difference between them.
 // DICTIONARY_ARENA_WASTE_BYTES remains a genuine nested waste figure for the
 // DICTIONARY arena. None of these nested counters are summed in here.
+//
+// The exported per-category counter names (native_mem_{live,avg,max,
+// chunk_overhead,post_flush_live,post_flush_max}_bytes.<category>, written by
+// Recording::writeNativeMem()) derive from this table, and the set grows when
+// a category is added, so consumers should match native_mem_*_bytes. counters
+// by prefix rather than hard-code the category list.
 #define DD_NATIVE_MEM_CATEGORY_TABLE(X)                                        \
   X(CALLTRACE, "calltrace")                                                    \
   X(DICTIONARY, "dictionary")                                                  \
@@ -38,6 +53,7 @@
   X(METHOD_MAP, "method_map")                                                  \
   X(LIVENESS, "liveness")                                                      \
   X(WALLCLOCK, "wallclock")                                                    \
+  X(CTIMER, "ctimer")                                                          \
   X(MISC, "misc")
 
 #define X_NM_ENUM(a, b) NM_##a,

ddprof-lib/src/main/cpp/perfEvents.h

@@ -52,6 +52,7 @@ class PerfEvents : public Engine {
 
   virtual int registerThread(int tid);
   virtual void unregisterThread(int tid);
+  void releaseThreads();
   long interval() const { return _interval; }
 
   const char *name() { return "PerfEvents"; }

ddprof-lib/src/main/cpp/perfEvents_linux.cpp

@@ -962,16 +962,29 @@ Error PerfEvents::start(Arguments &args) {
     // sites); capture the old size first since _max_events is overwritten below.
     long long old_bytes =
         (_events != NULL) ? (long long)((size_t)_max_events * sizeof(PerfEvent)) : 0;
+    // The table is the only record of which fd and ring belong to which tid
+    // (stop() keeps them). Release them while it still exists; otherwise
+    // they leak, and stay armed, once it is freed.
+    releaseThreads();
     free(_events);
     if (old_bytes > 0) {
       NativeMem::record(NM_PERF, -old_bytes);
     }
     _events = (PerfEvent *)calloc(max_events, sizeof(PerfEvent));
-    _max_events = max_events;
-    if (_events != NULL) {
-      NativeMem::record(NM_PERF,
-                        (long long)((size_t)max_events * sizeof(PerfEvent)));
+    if (_events == NULL) {
+      // Back to the not-started value, so registerThread() and
+      // unregisterThread() reject every tid. Return before the signal
+      // handler, the hook and the registration gate are set up: start()
+      // fails, so the CPU engine is neither registered nor enabled for this
+      // recording. The next start() retries the allocation.
+      _max_events = -1;
+      Log::error("perfEvents: could not allocate the table for %d threads; "
+                 "PerfEvents is disabled for this recording", max_events);
+      return Error("Failed to allocate perf event table");
     }
+    _max_events = max_events;
+    NativeMem::record(NM_PERF,
+                      (long long)((size_t)max_events * sizeof(PerfEvent)));
   }
 
   OS::installSignalHandler(SIGPROF, signalHandler);
@@ -980,6 +993,10 @@ Error PerfEvents::start(Arguments &args) {
   __atomic_store_n(_pthread_entry, (void *)pthread_setspecific_hook,
                    __ATOMIC_RELEASE);
 
+  // _events is sized: open the hook gate before listing threads (see
+  // Profiler::openThreadRegistration()).
+  Profiler::instance()->openThreadRegistration();
+
   // Create perf_events for all existing threads
   int threads_sz = 0, threads_cap;
   int *threads = (int *)malloc((threads_cap = 1024) * sizeof(int));
@@ -1052,12 +1069,28 @@ void PerfEvents::stop() {
   // check whether the thread has been registered already on start.
 }
 
+// stop() keeps every fd and ring, and only unregisterThread() releases them
+// when a thread exits. Releases them all when nothing else could:
+// Profiler::replaceCpuEngine() calls this under a ThreadTablesFreeze once
+// another CPU engine replaces this one, and start() calls it before it frees
+// the table on a pid_max change. The start() call runs without a freeze, so a
+// concurrent unregisterThread() of the same tid is still possible there
+// (PROF-16359).
+void PerfEvents::releaseThreads() {
+  for (int tid = 0; tid < _max_events; tid++) {
+    unregisterThread(tid);
+  }
+}
+
 int PerfEvents::walkKernel(int tid, const void **callchain, int max_depth,
                            StackContext *java_ctx) {
   if (!(_ring & RING_KERNEL)) {
     // we are not capturing kernel stacktraces
     return 0;
   }
+  if (_events == NULL || tid < 0 || tid >= _max_events) {
+    return 0; // no table: the allocation failed (see start())
+  }
 
   PerfEvent *event = &_events[tid];
   if (!event->tryLock()) {
@@ -1148,6 +1181,12 @@ int PerfEvents::walkKernel(int tid, const void **callchain, int max_depth,
 }
 
 void PerfEvents::resetBuffer(int tid) {
+  // Fds left armed by an earlier recording keep delivering SIGPROF, and the
+  // handler calls this on every exit, also after a failed table allocation
+  // left _events NULL (_max_events -1).
+  if (_events == NULL || tid < 0 || tid >= _max_events) {
+    return;
+  }
   PerfEvent *event = &_events[tid];
   if (!event->tryLock()) {
     return; // the event is being destroyed