Tracer Version(s)
1.64.0
Java Version(s)
21.0.11
JVM Vendor
Amazon Corretto
Bug Report
After upgrading dd-trace-java from 1.63.2 to 1.64.0, our Spring Boot applications (Tomcat based) started running out of Metaspace within about a day of deployment. Metaspace grows continuously in a straight line rather than plateauing, until it hits the configured limit and the JVM/pod is killed.
Downgrading to 1.63.2 with no other changes resolves the issue. On 1.64.0, setting DD_APPSEC_SCA_ENABLED=false also resolves it (all other flags unchanged). This strongly points at the SCA reachability work shipped in 1.64.0 (possibly #11352 "Implement SCA Reachability runtime detection" and/or #11614 "Migrate SCA Reachability to method-level symbol database") as the likely root cause, but we haven't been able to confirm the exact mechanism from our side.
I created a Datadog Helpdesk issue for this as well at Request #2955022 (with uploaded class load file).
Expected Behavior
Expected behavior
Metaspace usage should stabilize after the application warms up (JIT/class loading settles), as it did on 1.63.2 and as it does on 1.64.0 when Datadog instrumentation is fully disabled.
Actual behavior
Metaspace (jvm.gc.metaspace_size) grows continuously and roughly linearly for the life of the process. Example from one affected pod:
15:00 — Metaspace: ~110 MB
06:00 next day — Metaspace: ~200 MB
Growth does not plateau; left running, the process eventually hits the configured Metaspace limit and OOMs. This started the same day we rolled out a new image built against 1.64.0 — no other code or config changes shipped alongside it.
Reproduction Code
Run a Spring Boot (Tomcat) service on Java 21 with dd-java-agent 1.64.0 and the configuration above (in particular DD_APPSEC_SCA_ENABLED=true).
Let it run under normal traffic for several hours while monitoring jvm.gc.metaspace_size (or jcmd VM.metaspace / a JFR/heap capture).
Observe continuous, non-plateauing Metaspace growth until the container is OOM-killed (roughly within 24h in our environment, depending on Metaspace limit and traffic).
Tracer Version(s)
1.64.0
Java Version(s)
21.0.11
JVM Vendor
Amazon Corretto
Bug Report
After upgrading dd-trace-java from 1.63.2 to 1.64.0, our Spring Boot applications (Tomcat based) started running out of Metaspace within about a day of deployment. Metaspace grows continuously in a straight line rather than plateauing, until it hits the configured limit and the JVM/pod is killed.
Downgrading to 1.63.2 with no other changes resolves the issue. On 1.64.0, setting DD_APPSEC_SCA_ENABLED=false also resolves it (all other flags unchanged). This strongly points at the SCA reachability work shipped in 1.64.0 (possibly #11352 "Implement SCA Reachability runtime detection" and/or #11614 "Migrate SCA Reachability to method-level symbol database") as the likely root cause, but we haven't been able to confirm the exact mechanism from our side.
I created a Datadog Helpdesk issue for this as well at Request #2955022 (with uploaded class load file).
Expected Behavior
Expected behavior
Metaspace usage should stabilize after the application warms up (JIT/class loading settles), as it did on 1.63.2 and as it does on 1.64.0 when Datadog instrumentation is fully disabled.
Actual behavior
Metaspace (jvm.gc.metaspace_size) grows continuously and roughly linearly for the life of the process. Example from one affected pod:
15:00 — Metaspace: ~110 MB
06:00 next day — Metaspace: ~200 MB
Growth does not plateau; left running, the process eventually hits the configured Metaspace limit and OOMs. This started the same day we rolled out a new image built against 1.64.0 — no other code or config changes shipped alongside it.
Reproduction Code
Run a Spring Boot (Tomcat) service on Java 21 with dd-java-agent 1.64.0 and the configuration above (in particular DD_APPSEC_SCA_ENABLED=true).
Let it run under normal traffic for several hours while monitoring jvm.gc.metaspace_size (or jcmd VM.metaspace / a JFR/heap capture).
Observe continuous, non-plateauing Metaspace growth until the container is OOM-killed (roughly within 24h in our environment, depending on Metaspace limit and traffic).