[python] Add APPSEC_OTLP_EXPORT scenarios - #7578
Conversation
… (APPSEC-68836) Validate M2 topology: DD Tracer exporting traces via OTLP + DD Agent for Remote Config. New scenarios: - APPSEC_OTLP_EXPORT: blocking ruleset, tests WAF blocking + detection - APPSEC_OTLP_EXPORT_DEFAULT_RULES: default ruleset, tests alpha/obfuscator/payment OTLP-to-Datadog span adapter: - DataDogLibraryTraceOTLP/DataDogLibrarySpanOTLP wraps OTLP spans - Numeric coercion for metric keys (string -> int/float) - Base64+msgpack decoding for appsec trigger data - Timestamp mapping (startTimeUnixNano -> start/duration) - RID matching via _dd.top_level coercion Library interface fallback: - get_traces() falls back to interfaces.open_telemetry when no DD-format traces - get_appsec_events() decodes base64 appsec data from OTLP Test results: 109 passed, 0 failed, 4 xfailed across both scenarios. CI: scenarios run on tracer release PRs and when test files are modified.
|
|
RamyElkest
changed the title
|
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e75e9866a5
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| "OTEL_EXPORTER_OTLP_TRACES_ENDPOINT": f"http://proxy:{ProxyPorts.open_telemetry_weblog}/v1/traces", | ||
| "OTEL_EXPORTER_OTLP_TRACES_HEADERS": "dd-protocol=otlp,dd-otlp-path=agent", |
There was a problem hiding this comment.
In both new scenarios, dd-otlp-path=agent makes the proxy forward /v1/traces directly to agent:4318 (utils/proxy/core.py lines 227–237); the scenario also never creates an OpenTelemetryCollectorContainer. Consequently these tests validate DD Tracer → DD Agent OTLP ingestion, not the advertised M2 topology through a native OTel Collector, so collector-specific AppSec regressions remain completely untested.
Useful? React with 👍 / 👎.
| @scenarios.appsec_otlp_export | ||
| class Test_AppSec_OTLP_Blocking: |
There was a problem hiding this comment.
All four classes added in this file have only the scenario decorator and no @features.<feature> or @features.not_reported decorator. This violates the repository's required feature association and causes tests/test_the_test/test_features.py::test_all_class_has_feature_decorator to reject the collected suite, in addition to omitting these tests from feature reporting. .cursor/rules/repository-structure.mdcL83-L86
Useful? React with 👍 / 👎.
| spans = [] | ||
| for scope_span in resource_span.get("scopeSpans", []): | ||
| for raw_span in scope_span.get("spans", []): | ||
| spans.append(DataDogLibrarySpanOTLP(self, raw_span, resource_attrs)) | ||
| self.spans = spans |
There was a problem hiding this comment.
An OTLP ResourceSpans groups spans by resource and may contain spans from multiple trace IDs, but this constructor wraps every span in the resource as one DataDogLibraryTraceOTLP and derives its trace ID from the first span. Whenever the exporter batches requests, later spans therefore report the wrong span["trace_id"], and callers using full_trace=True or iterating the returned trace receive unrelated traces; group the spans by traceId before constructing each trace.
Useful? React with 👍 / 👎.
| def trace_id_as_int(self) -> int: | ||
| tid = self.trace_id | ||
| if isinstance(tid, str) and tid: | ||
| return int(tid, 16) & 0xFFFFFFFFFFFFFFFF |
There was a problem hiding this comment.
For OTLP sent as binary protobuf, MessageToDict represents the 16-byte traceId as base64 rather than hexadecimal (the existing _trace_id_to_hex helper in tests/otel/test_tracing_otlp.py explicitly handles both encodings). Because these scenarios do not force JSON transport, trace_id_as_int raises ValueError whenever trace_id_equals() or another existing trace-ID validator encounters a normal binary OTLP payload; decode the base64 form before applying the lower-64-bit mask.
Useful? React with 👍 / 👎.
| for data in interfaces.open_telemetry.get_data(path_filters=["/v1/traces", "/api/v0.2/traces"]): | ||
| content = data.get("request", {}).get("content", {}) | ||
| text = json.dumps(content) | ||
| for m in re.finditer(r'"([^"]*appsec[^"]*)"\s*:', text, re.IGNORECASE): | ||
| appsec_keys.add(m.group(1)) |
There was a problem hiding this comment.
Both OTLP search helpers scan every payload accumulated by interfaces.open_telemetry instead of filtering by the RID from each test's self.r. After any earlier setup exports an AppSec event, later tests can pass using that stale span even when their own request was not exported or lacks the required attributes, masking request-specific regressions; pass the current response into these helpers and inspect only its matching OTLP spans.
Useful? React with 👍 / 👎.
| if not trace_found: | ||
| # Fallback: try OTLP data from interfaces.open_telemetry | ||
| # This allows tests to work when traces are exported via OTLP instead of Datadog agent protocol | ||
| yield from self._get_traces_from_otlp(rid) |
There was a problem hiding this comment.
The OTLP adapter is used only when no matching Datadog-protocol trace was found. If a tracer ignores the new OTLP exporter setting, lacks support for it, or emits both formats, every reused AppSec assertion silently validates the ordinary native trace and the scenario remains green without proving that any AppSec data survived OTLP export; these OTLP-specific scenarios should prefer or explicitly require a matching trace from interfaces.open_telemetry.
Useful? React with 👍 / 👎.
…nts, IAST, metastruct (APPSEC-68836) Added @scenarios.appsec_otlp_export_default_rules to 73 additional test classes: - WAF rules (test_rules.py): 13 classes — Scanners, SQLI, XSS, LFI, SSRF, etc. - WAF addresses (test_addresses.py): 12 classes — UrlQuery, Headers, Cookies, BodyJson, etc. - Login events (test_event_tracking.py, test_automated_login_events.py): 9 classes - User identification (test_identify.py): 1 class - IAST (test_hardcoded_secrets.py, test_hsts_missing_header.py, etc.): 16 classes - Metastruct (test_metastruct.py): 2 classes - Misc (test_miscs.py, test_truncation.py, test_conf.py): 5 classes - Reports (test_reports.py): 2 classes - Agentic onboarding: 1 class Adapter fixes: - Added 'iast' and '_dd.iast.json' to _APPSEC_DATA_KEYS for base64 decoding - Added 'service' field mapping from OTLP resource attribute 'service.name' Test results: 177 passed, 25 skipped, 36 xfailed, 0 failed.
RamyElkest
changed the title
Added tests/appsec/smoke_tests/test_otlp_export.py with: - Threats smoke tests (WAF attack detection via agent interface) - UserEvents smoke tests (user login event tracking via agent interface) Both run in APPSEC_OTLP_EXPORT and APPSEC_OTLP_EXPORT_DEFAULT_RULES. RASP, RemoteConfig, and ApiSecurity smoke tests require additional scenario configuration not yet available in OTLP export scenarios. Test results: 4 smoke tests passed, 0 failed.
- Fix mypy list invariance: type annotate spans as list[DataDogLibrarySpan] - Add @features.not_reported to spike test classes (required by test_all_class_has_feature_decorator)
RamyElkest
changed the title
RamyElkest
changed the title
The spike tests were diagnostic tools to validate the OTLP adapter during development. They fail on non-Python languages (Ruby, Node.js) that don't support OTLP export. The real test coverage is in the 177 tests added to existing test files.
1 participant
Summary
Validates AAP for DD Tracer exporting traces via OTLP + DD Agent
Closes APPSEC-68836
What's New
Two Scenarios
APPSEC_OTLP_EXPORTAPPSEC_OTLP_EXPORT_DEFAULT_RULESBoth use:
include_opentelemetry=True+include_agent=True+rc_api_enabled=True+appsec_enabled=TrueOTLP-to-Datadog Span Adapter (
_datadog_library_trace_otlp.py)Wraps OTLP spans into the
DataDogLibrarySpaninterface so existing AppSec tests work unchanged:resourceSpans→scopeSpans→spansstartTimeUnixNano/endTimeUnixNano→start/duration_dd.top_leveland_sampling_priority_v1for RID matchingLibrary Interface Fallback (
_library/core.py)get_traces()falls back tointerfaces.open_telemetrywhen no Datadog-format traces foundget_appsec_events()decodes base64 appsec data via_decode_appsec_data()Test Results
APPSEC_OTLP_EXPORTAPPSEC_OTLP_EXPORT_DEFAULT_RULESCI Integration
.github/workflows/run-end-to-end.ymltracer_releasegroup — runs on every tracer release PR@scenarios.appsec_otlp_exportare modifiedFiles Changed
utils/dd_types/_datadog_library_trace_otlp.py— OTLP span adaptertests/appsec/test_appsec_otlp_spike.py— spike validation testsutils/_context/_scenarios/__init__.py— 2 new scenario definitionsutils/interfaces/_library/core.py— OTLP fallback + appsec decoderutils/dd_types/__init__.py— export new classestests/appsec/—@scenariosdecorators.github/workflows/run-end-to-end.yml— CI stepstests/test_the_test/test_group_rules.py— exclusion list