Skip to content

Comments

Push llm event address#3664

Open
estringana wants to merge 10 commits intomasterfrom
estringana/add-openai-integration
Open

Push llm event address#3664
estringana wants to merge 10 commits intomasterfrom
estringana/add-openai-integration

Conversation

@estringana
Copy link
Contributor

@estringana estringana commented Feb 19, 2026

Description

This PR enables appsec capabilities when using openai-php client. The implementation push appsec addresses to the waf and then eventually they are reported to the backend.

More info on RFC: API Endpoints: AI usage

Reviewer checklist

  • Test coverage seems ok.
  • Appropriate labels assigned.

@datadog-official
Copy link

datadog-official bot commented Feb 19, 2026

⚠️ Tests

Fix all issues with BitsAI or with Cursor

⚠️ Warnings

🧪 1044 Tests failed

testSearchPhpBinaries from integration.DDTrace\Tests\Integration\PHPInstallerTest (Datadog) (Fix with Cursor)
Risky Test
phpvfscomposer://tests/vendor/phpunit/phpunit/phpunit:52
testSimplePushAndProcess from laravel-58-test.DDTrace\Tests\Integrations\Laravel\V5_8\QueueTest (Datadog) (Fix with Cursor)
DDTrace\Tests\Integrations\Laravel\V5_8\QueueTest::testSimplePushAndProcess
Test code or tested code printed unexpected output: spanLinksTraceId: 699d9af000000000c1d2f93fd989b610
tid: 699d9af000000000
hexProcessTraceId: c1d2f93fd989b610
hexProcessSpanId: 16c1092892aa0f46
processTraceId: 13966499447057266192
processSpanId: 1639601809203531590

phpvfscomposer://tests/vendor/phpunit/phpunit/phpunit:106
testSimplePushAndProcess from laravel-8x-test.DDTrace\Tests\Integrations\Laravel\V8_x\QueueTest (Datadog) (Fix with Cursor)
DDTrace\Tests\Integrations\Laravel\V8_x\QueueTest::testSimplePushAndProcess
Test code or tested code printed unexpected output: spanLinksTraceId: 699d9b8d000000007d2b5e2e9dff4e59
tid: 699d9b8d00000000
hexProcessTraceId: 7d2b5e2e9dff4e59
hexProcessSpanId: d92487cb616fefcc
processTraceId: 9019406233051811417
processSpanId: 15646780312974782412
View all

ℹ️ Info

❄️ No new flaky tests detected

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 86bde1a | Docs | Datadog PR Page | Was this helpful? Give us feedback!

@codecov-commenter
Copy link

codecov-commenter commented Feb 19, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 62.10%. Comparing base (a3409cd) to head (86bde1a).

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #3664      +/-   ##
==========================================
- Coverage   62.22%   62.10%   -0.12%     
==========================================
  Files         141      141              
  Lines       13352    13352              
  Branches     1746     1746              
==========================================
- Hits         8308     8292      -16     
- Misses       4253     4266      +13     
- Partials      791      794       +3     

see 3 files with indirect coverage changes


Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a3409cd...86bde1a. Read the comment docs.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@pr-commenter
Copy link

pr-commenter bot commented Feb 19, 2026

Benchmarks [ tracer ]

Benchmark execution time: 2026-02-24 13:43:03

Comparing candidate commit 86bde1a in PR branch estringana/add-openai-integration with baseline commit a3409cd in branch master.

Found 6 performance improvements and 32 performance regressions! Performance is the same for 155 metrics, 1 unstable metrics.

scenario:ComposerTelemetryBench/benchTelemetryParsing

  • 🟥 mem_peak [+87.128KB; +87.128KB] or [+2.181%; +2.181%]

scenario:ContextPropagationBench/benchExtractHeaders128Bit

  • 🟥 mem_peak [+98.840KB; +98.840KB] or [+2.474%; +2.474%]

scenario:ContextPropagationBench/benchExtractHeaders64Bit

  • 🟥 mem_peak [+98.840KB; +98.840KB] or [+2.474%; +2.474%]

scenario:ContextPropagationBench/benchExtractTraceContext128Bit

  • 🟥 mem_peak [+98.840KB; +98.840KB] or [+2.474%; +2.474%]

scenario:ContextPropagationBench/benchExtractTraceContext64Bit

  • 🟥 mem_peak [+98.840KB; +98.840KB] or [+2.474%; +2.474%]

scenario:ContextPropagationBench/benchInject128Bit

  • 🟥 mem_peak [+98.848KB; +98.848KB] or [+2.474%; +2.474%]

scenario:ContextPropagationBench/benchInject64Bit

  • 🟥 mem_peak [+98.848KB; +98.848KB] or [+2.474%; +2.474%]

scenario:EmptyFileBench/benchEmptyFileBaseline-opcache

  • 🟥 execution_time [+132.348µs; +410.552µs] or [+3.997%; +12.400%]

scenario:HookBench/benchHookOverheadInstallHookOnFunction

  • 🟥 mem_peak [+98.712KB; +98.712KB] or [+2.470%; +2.470%]

scenario:HookBench/benchHookOverheadInstallHookOnMethod

  • 🟥 mem_peak [+98.712KB; +98.712KB] or [+2.470%; +2.470%]

scenario:HookBench/benchHookOverheadTraceFunction

  • 🟥 mem_peak [+107.551KB; +107.554KB] or [+2.421%; +2.421%]

scenario:HookBench/benchHookOverheadTraceMethod

  • 🟥 mem_peak [+107.551KB; +107.556KB] or [+2.387%; +2.387%]

scenario:HookBench/benchWithoutHook

  • 🟥 mem_peak [+98.728KB; +98.728KB] or [+2.471%; +2.471%]

scenario:LogsInjectionBench/benchLogsInfoInjection-opcache

  • 🟩 execution_time [-622.549ns; -244.051ns] or [-6.996%; -2.743%]

scenario:MessagePackSerializationBench/benchMessagePackSerialization

  • 🟥 mem_peak [+107.552KB; +107.552KB] or [+2.529%; +2.529%]

scenario:MessagePackSerializationBench/benchMessagePackSerialization-opcache

  • 🟩 execution_time [-6.618µs; -5.542µs] or [-5.916%; -4.954%]

scenario:PDOBench/benchPDOBaseline

  • 🟥 mem_peak [+107.552KB; +107.552KB] or [+2.667%; +2.667%]

scenario:PHPRedisBench/benchRedisBaseline

  • 🟥 mem_peak [+97.968KB; +97.968KB] or [+2.452%; +2.452%]

scenario:SamplingRuleMatchingBench/benchGlobMatching1

  • 🟥 mem_peak [+101.968KB; +101.968KB] or [+2.552%; +2.552%]

scenario:SamplingRuleMatchingBench/benchGlobMatching2

  • 🟥 mem_peak [+101.968KB; +101.968KB] or [+2.552%; +2.552%]

scenario:SamplingRuleMatchingBench/benchGlobMatching3

  • 🟥 mem_peak [+101.968KB; +101.968KB] or [+2.552%; +2.552%]

scenario:SamplingRuleMatchingBench/benchGlobMatching4

  • 🟥 mem_peak [+101.968KB; +101.968KB] or [+2.552%; +2.552%]

scenario:SamplingRuleMatchingBench/benchRegexMatching1

  • 🟥 execution_time [+104.686ns; +148.714ns] or [+9.093%; +12.917%]
  • 🟥 mem_peak [+101.968KB; +101.968KB] or [+2.552%; +2.552%]

scenario:SamplingRuleMatchingBench/benchRegexMatching1-opcache

  • 🟩 execution_time [-120.726ns; -67.274ns] or [-8.689%; -4.842%]

scenario:SamplingRuleMatchingBench/benchRegexMatching2

  • 🟥 execution_time [+100.739ns; +149.861ns] or [+8.714%; +12.963%]
  • 🟥 mem_peak [+101.968KB; +101.968KB] or [+2.552%; +2.552%]

scenario:SamplingRuleMatchingBench/benchRegexMatching2-opcache

  • 🟩 execution_time [-132.762ns; -84.438ns] or [-9.476%; -6.027%]

scenario:SamplingRuleMatchingBench/benchRegexMatching3

  • 🟥 execution_time [+97.316ns; +149.084ns] or [+8.308%; +12.727%]
  • 🟥 mem_peak [+101.968KB; +101.968KB] or [+2.552%; +2.552%]

scenario:SamplingRuleMatchingBench/benchRegexMatching3-opcache

  • 🟩 execution_time [-129.660ns; -68.540ns] or [-9.238%; -4.883%]

scenario:SamplingRuleMatchingBench/benchRegexMatching4

  • 🟥 execution_time [+111.415ns; +170.785ns] or [+9.630%; +14.761%]
  • 🟥 mem_peak [+101.968KB; +101.968KB] or [+2.552%; +2.552%]

scenario:SamplingRuleMatchingBench/benchRegexMatching4-opcache

  • 🟩 execution_time [-117.645ns; -54.355ns] or [-8.466%; -3.912%]

scenario:SpanBench/benchDatadogAPI

  • 🟥 mem_peak [+100.200KB; +100.200KB] or [+2.508%; +2.508%]

scenario:TraceAnnotationsBench/benchTraceAnnotationOverhead

  • 🟥 mem_peak [+107.552KB; +107.552KB] or [+2.383%; +2.383%]

scenario:TraceFlushBench/benchFlushTrace

  • 🟥 mem_peak [+107.551KB; +107.552KB] or [+2.638%; +2.638%]

scenario:TraceSerializationBench/benchSerializeTrace

  • 🟥 mem_peak [+107.551KB; +107.554KB] or [+2.578%; +2.578%]

@estringana estringana force-pushed the estringana/add-openai-integration branch 2 times, most recently from cff1eed to 1ae39aa Compare February 20, 2026 12:06
@estringana estringana marked this pull request as ready for review February 22, 2026 14:57
@estringana estringana requested review from a team as code owners February 22, 2026 14:57
@estringana estringana force-pushed the estringana/add-openai-integration branch from f363215 to 3fadba0 Compare February 23, 2026 09:55
@estringana estringana force-pushed the estringana/add-openai-integration branch from 3fadba0 to 6028af2 Compare February 23, 2026 09:55
Copy link
Contributor

@cataphract cataphract left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd wait for a production rule to be ready (if it ain't already), update the recommended.json files and then also write an integration test. This would validate the correctness of the address and its parameters.

@estringana
Copy link
Contributor Author

I'd wait for a production rule to be ready (if it ain't already), update the recommended.json files and then also write an integration test. This would validate the correctness of the address and its parameters.

I have the rule https://github.com/DataDog/appsec-event-rules/pull/265 but it's not merged yet. Also we would need to mock the openai library http call. Do we have a system for that already on integration?
@cataphract

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants