Skip to content

fix(drupal): hook the theme engine service and guarantee render hook removal - #4145

Open
Leiyks wants to merge 7 commits into
masterfrom
leiyks/fix-apms-20395
Open

fix(drupal): hook the theme engine service and guarantee render hook removal#4145
Leiyks wants to merge 7 commits into
masterfrom
leiyks/fix-apms-20395

Conversation

@Leiyks

@Leiyks Leiyks commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

Description

Fixes APMS-20395.

On Drupal >= 11.3, drupal.template.file was never set on any drupal.theme.render span, and hooks accumulated on twig_render_template until DD_TRACE_HOOK_LIMIT. A customer measured the tag present on 0 of 61,160 render spans.

Root cause. Drupal 11.3.0 switched ThemeManager::render() to a theme-engine service ([$engine, 'renderTemplate']), falling back to the global {engine}_render_template() only when no service is registered. Core ships a Twig service, so the global is never called — but it is still defined, because ThemeInitialization unconditionally includes twig.engine. So function_exists() returned true, the integration installed a hook on every render, the hook never fired, and it was never removed.

A second defect, on every Drupal version: render() can return early (theme hook not found, exception) without calling the render function. Those hooks leaked and then fired on later renders, overwriting unrelated spans' tags — measured 19 spans mis-tagged in a 30-render test.

Changes

  • Hook ThemeEngineInterface::renderTemplate once at init() — covers Twig, contrib engines, and module templates in one install. The service path passes the template path without its extension, so .html.twig is re-appended for Twig to keep the tag value identical to pre-11.3.
  • Install the legacy {engine}_render_template hook once per engine instead of once per render, seeded on twig_render_template at init(). This also fixes a pre-existing gap: core falls back to twig_render_template when {engine}_render_template is absent, which was previously untagged.
  • Create the drupal.theme.render span from install_hook + $hook->span() rather than trace_method. $hook->span() is per-invocation, so the frame's own span is known directly — no active_span() lookup and no guessing which frame a tag belongs to.
  • Bail early when dd_trace_tracer_is_limited(). Unlike trace_method, install_hook callbacks are not limit-gated, so without this the whole body ran on every render past the span limit (measured 200x on a 200-render probe).

Reviewer notes

  • allowNestedHook() is not needed: it escapes reentrancy initiated from inside a callback, not the hooked function recursing. install_hook holds running only across the callback, so it is already recursive — measured 7/7 begins on a nested render chain, versus 1/1 for trace_method with recurse => false.
  • Tag attribution is by frame, not by span name — an ancestor render span carries the same name under recursion, so a name check would mis-attribute a nested template to its parent.

@datadog-official

datadog-official Bot commented Aug 27, 2026

Copy link
Copy Markdown

Pipelines  Tests

Unblock PR with BitsAI

⚠️ Warnings

Your PR has failed checks. Please review the issues below and take necessary action before merging.

🚦 4 Pipeline jobs failed

DataDog/apm-reliability/dd-trace-php | test_extension_ci: [8.3] — 🔧 Needs a code fix, caused by this PR

View more details · View in GitLab

DataDog/apm-reliability/dd-trace-php | ASAN test_c with multiple observers: [8.3]

View more details · View in GitLab

DataDog/apm-reliability/dd-trace-php | publish docker image for system tests

View more details · View in GitLab

View all 4 failed jobs.

ℹ️ Info

No other issues found (see more)

🧪 All tests passed
❄️ No new flaky tests detected

🎯 Code Coverage (details)
Patch Coverage: 100.00%
Overall Coverage: 60.60% (-0.07%)

Useful? React with 👍 / 👎

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 730f721 | Docs | View more details | Give us feedback!

@pr-commenter

pr-commenter Bot commented Aug 27, 2026

Copy link
Copy Markdown

Benchmarks [ tracer ]

Benchmark execution time: 2026-08-31 13:30:50

Comparing candidate commit 730f721 in PR branch leiyks/fix-apms-20395 with baseline commit f4c0c7e in branch master.

Found 1 performance improvements and 0 performance regressions! Performance is the same for 192 metrics, 1 unstable metrics.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

  • 🟩 = significantly better candidate vs. baseline
  • 🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

scenario:MessagePackSerializationBench/benchMessagePackSerialization-opcache

  • 🟩 execution_time [-4.110µs; -2.790µs] or [-3.534%; -2.399%]

@Leiyks
Leiyks marked this pull request as ready for review August 28, 2026 14:25
@Leiyks
Leiyks requested review from a team as code owners August 28, 2026 14:25
@Leiyks
Leiyks requested review from LobeTia and removed request for a team August 28, 2026 14:25
Comment thread src/DDTrace/Integrations/Drupal/DrupalIntegration.php Outdated
Leiyks added 4 commits August 31, 2026 11:39
…removal

Drupal 11.3 moved template rendering to a theme_engine service
(ThemeEngineInterface::renderTemplate), and ThemeManager::render() only falls
back to the deprecated {engine}_render_template() global when no such service
resolves. Core still includes twig.engine, so function_exists() stays true
while the function is never called: the integration installed a hook per
render that never fired and never self-removed, so drupal.template.file was
missing on every drupal.theme.render span and datadog.trace.hook_limit was
reached within a single request.

Hook ThemeEngineInterface::renderTemplate once at init() to cover every engine
implementation, re-appending .html.twig for TwigThemeEngine so the tag keeps
its pre-11.3 value, and take the legacy per-render branch only when no engine
service resolves. The membership test goes through the themeEngines service
collection rather than getThemeEngine(), which would instantiate the engine
even on core's early-return path.

install_hook's callback is not gated by the span limit while trace_method is,
so past the limit a nested render has no span of its own and active_span()
returns the outer render's. Bail out when the tracer is limited rather than
tag that span with the nested template.

Independently, ThemeManager::render() can return without ever calling the
render function (unknown theme hook, exception) on every Drupal version, so
the callback's self-removal is no longer the only removal path: the posthook
now removes the id the prehook recorded for that span. Keyed by span rather
than a stack because the posthook is skipped for a dropped span.

APMS-20395
The ThemeManager::render prehook reset $renderHookIds[$spanKey] to 0
unconditionally. A dropped span skips the posthook (dd_uhook_end gates
the end hook on !dyn->dropped_span in tracer/hook/uhook_legacy.c), so
the slot could still hold a live, installed hook id. Once the freed
SpanData's object handle was recycled, spl_object_hash collided and the
reset discarded that id without removing the hook, orphaning it for the
rest of the request.

Remove the hook before resetting the slot. remove_hook() on an already
removed id is a no-op, so the common self-removal path is unaffected.

APMS-20395
Per bwoebi's review on #4145: use install_hook() and pass the legacy
{engine}_render_template() hook id through $hook->data instead of keeping a
map keyed by spl_object_hash($span). $hook->data is per-invocation, so
recursion needs no keying at all, and spl_object_hash (deprecated in PHP 8.6)
is gone without reaching for spl_object_id, which is PHP 7.2+ while this
package supports PHP 7.0.

The install_hook end hook also runs where the tracing posthook does not: it is
gated only on the begin hook having run (tracer/hook/uhook.c:424-430), whereas
the tracing posthook is skipped for a dropped span
(tracer/hook/uhook_legacy.c:208-226) and for a span-limited call
(uhook_legacy.c:102-105). Removal is therefore unconditional rather than
best-effort, which theme_render_dropped_span.phpt now covers.

install_hook callbacks are not span-limit gated while trace_method is, so the
tag write keeps the active-span guard, now shared by both engine paths;
theme_render_span_limit.phpt guards the legacy path against the nested-render
mis-tagging that the guard prevents.

APMS-20395
The template tag was targeted with a `$span->name === 'drupal.theme.render'`
check, but under `'recurse' => true` an ancestor render span carries that same
name, so the check cannot tell this frame's span from an outer frame's. When a
nested render's span is hard-dropped after its tracing prehook ran, active_span()
reverts to the ancestor and the inner template overwrote the outer render's
correct tag.

This is not purely a regression from carrying the hook id via HookData::data:
the 11.3+ theme engine service path has been exposed since 4e02a93 added it,
because that hook has always resolved its target with active_span(). The
refactor widened the same latent bug to the legacy path.

Capture the frame's own span once in the ThemeManager::render begin hook and
compare by identity at the tag site, saving and restoring the enclosing value
through $hook->data so nesting needs no keyed storage. The install_hook is now
registered after the trace_method deliberately: begin hooks run in installation
order, so active_span() there is this render's own span. The span-limit check
stays, relocated to the capture point -- past the limit the frame has no span of
its own and active_span() is the parent's, which identity alone cannot reject.

APMS-20395
@Leiyks
Leiyks force-pushed the leiyks/fix-apms-20395 branch from b4c5969 to 5acdc24 Compare August 31, 2026 10:29
Comment thread src/DDTrace/Integrations/Drupal/DrupalIntegration.php Outdated
Leiyks added 2 commits August 31, 2026 13:46
Fold the ThemeManager::render trace_method into the install_hook that
already wrapped it, so a single hook owns the drupal.theme.render span.

allowNestedHook() is not needed: it escapes reentrancy initiated from a
hook callback (uhook.stub.php:106-110, uhook.c:1090-1103), not
self-recursion. install_hook has no persistent reentrancy guard --
def->running is set only around the callback (uhook.c:361-363,
:482-490) -- so it is unconditionally recursive. trace_method instead
holds def->active across the whole call (uhook_legacy.c:107 -> :250),
which is why it needed 'recurse' => true. Measured on a 3-deep plus
sibling nest: begin 4 / end 4 either way.

HookData::span() is keyed by invocation (uhook.c:870-905) and uses the
same allocator as trace_method (span.c:513-578), so it hands each frame
its own span. That removes the two guards the split design required:
dd_trace_tracer_is_limited() and the $renderSpan === active_span()
identity check. Past the span limit a nested render now gets a dummy
span that is never pushed (uhook.c:875-879) instead of no span at all,
so it can no longer leave active_span() on an ancestor -- and the outer
render, whose span was allocated before the limit was reached, keeps
its own template instead of losing the tag.

$tagTemplateFile and the $renderSpan save/restore via HookData::data
stay: the theme-engine hook fires in renderTemplate's frame, so its own
span() would be the wrong span.

The "spans out of sync" LOG_ONCE (uhook_legacy.c:216) can no longer be
emitted, as it lives on trace_method's dropped-span path; three tests
drop that expectation. The two span-limit tests now assert the outer
render keeps page.html.twig, and gained a probe proving the nested
render really ran so the assertion cannot pass vacuously.
The trace_method to install_hook conversion lost trace_method's cheap
bail: install_hook runs both callbacks past the span limit, where span()
only hands out a dummy. Over 200 renders past a 2-span limit that turned
1 getActiveTheme() call into 200 and 2 runtime registry lookups into 400.
The begin callback now returns early when the tracer is limited, clearing
$renderSpan so a limited frame's template cannot leak onto the enclosing
render's span; the end callback reads the frame span back from
$renderSpan and skips its work when the frame bailed out.

The {engine}_render_template hook was also installed and self-removed on
every render. Now that $renderSpan owns the attribution, one hook per
engine name for the whole request is enough, so the per-render
install/remove pair, the [$enclosing, $hookId] tuple and the themeEngines
discriminator are gone. Seeding twig_render_template at init() closes two
gaps: core falls back to twig's render function when the active engine has
no {engine}_render_template() of its own, and the deprecated global does
not delegate to the 11.3+ theme engine service.

Dropping the themeEngines discriminator also fixes a mismatch with core,
which resolves the engine of the template's own extension type rather
than the active theme's.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants