Run startup benchmarks on PRs

[sarahchen6]

What Does This Do

Run Startup benchmarks, defined in https://github.com/DataDog/apm-sdks-benchmarks/blob/main/.gitlab/ci-java-startup-parallel.yml, on every PR. Dacapo and Load benchmarks still need to be manually triggered to run on PRs.

Motivation

Startup benchmarks are the most easy to regress, so it is important we catch these as early as possible. Additionally, they do not run for very long (i.e. ~12 min), so running them on each PR should not slow down CI time.

Additional Notes

Contributor Checklist

• Format the title according to the contribution guidelines
• Assign the type: and (comp: or inst:) labels in addition to any other useful labels
• Avoid using close, fix, or any linking keywords when referencing an issue
  Use solves instead, and assign the PR milestone to the issue
• Update the CODEOWNERS file on source file addition, migration, or deletion
• Update public documentation with any new configuration flags or behaviors

Jira ticket: [PROJ-IDENT]

Note: Once your PR is ready to merge, add it to the merge queue by commenting /merge. /merge -c cancels the queue request. /merge -f --reason "reason" skips all merge queue checks; please use this judiciously, as some checks do not run at the PR-level. For more information, see this doc.

[dd-octo-sts]

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite	Status
Startup	🟢 pass

SLO thresholds are based on the latest master branch definitions. A warning is raised when results are within 5% of the threshold.

PR vs. master results

Startup Time

Scenario	This PR	master	Change
insecure-bank / iast	13,773 ms	13,719 ms	+0.4%
insecure-bank / tracing	12,722 ms	12,615 ms	+0.8%
petclinic / appsec	16,216 ms	15,768 ms	+2.8%
petclinic / iast	16,248 ms	15,819 ms	+2.7%
petclinic / profiling	16,295 ms	15,576 ms	+4.6%
petclinic / tracing	15,460 ms	15,077 ms	+2.5%

DaCapo Execution Time

Scenario	This PR	master	Change
biojava / appsec	13,904 ms	14,658 ms	-5.1%
biojava / baseline	14,688 ms	14,696 ms	-0.1%
biojava / iast	16,792 ms	17,565 ms	-4.4%
biojava / iast_GLOBAL	16,659 ms	17,361 ms	-4.0%
biojava / profiling	14,770 ms	14,654 ms	+0.8%
biojava / tracing	14,515 ms	14,578 ms	-0.4%
tomcat / appsec	2.20 ms	2.12 ms	+3.8%
tomcat / baseline	1.44 ms	1.44 ms	+0.0%
tomcat / iast	1.95 ms	1.86 ms	+5.0%
tomcat / iast_GLOBAL	1.93 ms	1.83 ms	+5.1%
tomcat / profiling	1.70 ms	1.70 ms	+0.3%
tomcat / tracing	1.73 ms	1.71 ms	+0.6%

Load Latency (p50)

Scenario	This PR	master	Change
insecure-bank / iast	1.19 ms	1.19 ms	+0.3%
insecure-bank / iast_FULL	2.57 ms	2.68 ms	-4.0%
insecure-bank / iast_GLOBAL	1.33 ms	1.28 ms	+4.2%
insecure-bank / no_agent	0.60 ms	0.59 ms	+0.7%
insecure-bank / profiling	0.90 ms	0.93 ms	-3.3%
insecure-bank / tracing	0.90 ms	0.83 ms	+9.1%
spring-petclinic / appsec	28.56 ms	28.92 ms	-1.2%
spring-petclinic / code_origins	24.72 ms	24.96 ms	-1.0%
spring-petclinic / iast	25.61 ms	26.20 ms	-2.3%
spring-petclinic / no_agent	23.68 ms	22.46 ms	+5.4%
spring-petclinic / profiling	28.17 ms	28.05 ms	+0.4%
spring-petclinic / tracing	25.19 ms	25.10 ms	+0.4%

_{Commit: 56bd20f9 · CI Pipeline · Benchmarking Platform UI}

Load and DaCapo benchmarks

Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.