Skip to content

feat: chaitin waf response logging - #13763

Open
blaisewang wants to merge 2 commits into
apache:masterfrom
blaisewang:feat/chaitin-waf-response-logging
Open

feat: chaitin waf response logging#13763
blaisewang wants to merge 2 commits into
apache:masterfrom
blaisewang:feat/chaitin-waf-response-logging

Conversation

@blaisewang

Copy link
Copy Markdown
Contributor

Description

lua-resty-t1k 1.2.0 can report the response to the SafeLine WAF detection service in addition to the request. This PR bumps the dependency and exposes the capability through the chaitin-waf plugin.

Until now the plugin only reported requests, so anything the WAF could have detected on the way out — data leaks in a response body, a successful exploit's output, an unexpected status code — was invisible to SafeLine. Enabling response reporting closes that gap without changing how requests are handled.

Three new config options, available both at plugin level and in the plugin metadata:

Option Type Default Description
log_resp boolean false Report the response to the WAF service.
resp_body_size integer (>= 0) 4 How much of the response body to report, in KB. 0 reports only the status line and headers.
extra_ignored_content_types string Comma separated response Content-Type values to skip, on top of the built-in list.

How it works:

  • The plugin now implements the body_filter and log phases, delegating to t1k.do_body_filter() and t1k.do_log().
  • body_filter buffers up to resp_body_size KB of the response body; the report itself is sent from an ngx.timer during the log phase, after the response has been handed back to the client. It therefore adds no latency to the response.
  • The report is advisory only: detection results appear in the SafeLine console, and APISIX never blocks or modifies a response based on them. Request handling is unchanged.
  • Responses are skipped when the request was already blocked, or when the response Content-Type is in the ignored list (audio, video, font, image and other binary media types out of the box).

One refactor was needed along the way: get_conf applied the metadata and plugin level config blocks with two verbatim copies of the same field-by-field assignment. Adding three more options would have meant keeping three copies in sync, so both are folded into a local helper. Precedence is unchanged — plugin level config still overrides metadata.

Docs updated in both en and zh, including a new "Response Logging" section that covers the async reporting model, the skip conditions, and the memory cost of buffering response bodies on routes serving large responses.

Which issue(s) this PR fixes:

N/A — no existing issue; this adds a new capability to the plugin.

Checklist

  • I have explained the need for this PR and the problem it solves
  • I have explained the changes or the new features added to this PR
  • I have added tests corresponding to this change
  • I have updated the documentation to reflect this change
  • I have verified that this change is backward compatible (If not, please discuss on the APISIX mailing list first)

blaisewang and others added 2 commits July 30, 2026 16:37
lua-resty-t1k 1.2.0 can report the response back to the SafeLine WAF
detection service in addition to the request. Bump the dependency and
expose it through three new `config` options:

- `log_resp` enables response reporting, off by default so existing
  configurations keep their current behavior
- `resp_body_size` caps how much of the response body is buffered and
  reported, in KB, defaulting to 4
- `extra_ignored_content_types` skips further response content types on
  top of the built-in ignored list

Wire up the `body_filter` and `log` phases to the corresponding library
entry points. The report is sent from an ngx.timer during the log phase,
so it does not delay the response, and the detection result is only
visible in the SafeLine console: a response is never blocked or modified
based on it.

`get_conf` applied the metadata and route level config blocks with two
copies of the same field-by-field assignment, which three more options
would have made harder to keep in sync. Fold both into a local helper,
keeping the existing precedence of route level config over metadata.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Add schema validation cases for the `resp_body_size` minimum and the
`extra_ignored_content_types` type, plus round-trip cases asserting that a
route configured with response logging still serves its response unchanged,
and that the default configuration behaves the same way.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@dosubot dosubot Bot added size:L This PR changes 100-499 lines, ignoring generated files. dependencies Pull requests that update a dependency file doc Documentation things enhancement New feature or request plugin labels Jul 30, 2026
Comment on lines +171 to +177
-- amount of the response body to report, in KB,
-- 0 disables body buffering
resp_body_size = {
type = "integer",
minimum = 0,
default = 4
},

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use bytes as the unit to maintain design consistency with other plugins.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the review — happy to change it.

One thing I want to check with you before I do, though: config.req_body_size in this plugin is already documented and shipped as "The maximum allowed request body size, in KB". It has been KB since the plugin was introduced in #9838, so it is part of the public API today.

That leaves two ways to go, and I'd rather you pick than guess:

  1. Only change resp_body_size to bytes. Consistent with the other plugins, but then the two body-size options inside the same config block use different units — req_body_size: 1024 means 1 MB while resp_body_size: 1024 means 1 KB. That seems more confusing for users of this plugin than the current inconsistency across plugins.
  2. Change both to bytes. Consistent everywhere, but req_body_size is an existing option, so this is a breaking change for anyone who has it configured: a route with req_body_size: 1024 would silently go from a 1 MB limit to a 1 KB limit. That would need some kind of migration/compat handling — e.g. renaming to max_req_body_bytes / max_resp_body_bytes (matching the *-logger plugins) and keeping req_body_size accepted as a deprecated alias.

@membphis membphis left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two P2 issues need to be addressed before merge:

  1. Response reporting can fail silently, and the tests do not verify the core behavior. ngx.timer.at does not consume callback return values, so connect/send/receive failures returned by socket.do_socket are neither logged nor measured. Tests 18-21 only verify configuration acceptance and the client response; they do not prove that the mock WAF receives the response status, headers, and truncated body, or that ignored content types are skipped. Please add observable error handling and an end-to-end assertion with bounded polling for the asynchronous report.

  2. The required exact-head CI is failing. The t/plugin/[a-k]*.t job is red in kafka-logger.t TEST 18 because the expected partition_id: 1 and partition_id: 2 log entries are missing. This appears outside the files changed by this PR, but the required check still needs to pass before merge. Please rerun it and investigate if the failure is reproducible: failed job.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dependencies Pull requests that update a dependency file doc Documentation things enhancement New feature or request plugin size:L This PR changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants