proxy: input plugins can now emit metrics and traces event types#11915
proxy: input plugins can now emit metrics and traces event types#11915seblaz wants to merge 1 commit into
Conversation
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (1)
🚧 Files skipped from review as they are similar to previous changes (1)
📝 WalkthroughWalkthroughThe proxy input collection callback now routes metrics, traces, and logs based on the proxy definition's event_type, adding msgpack decoder support for metrics/traces and calling the corresponding flb_input_*_append functions, with unified cleanup and error handling. ChangesEvent-Type Routing for Proxy Input Plugins
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes Possibly related PRs
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Warning There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure. 🔧 Infer (1.2.0)src/flb_plugin_proxy.csrc/flb_plugin_proxy.c:25:10: fatal error: 'monkey/mk_core.h' file not found ... [truncated 716 characters] ... /infer-linux-x86_64-v1.2.0/lib/infer/facebook-clang-plugins/clang/install/lib/clang/18/include" Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
seblaz
force-pushed
the
feat/proxy-input-event-type
branch
from
79248c1 to
59f97d1
Compare
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 79248c1980
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| if (event_type == FLB_INPUT_METRICS) { | ||
| ret = cmt_decode_msgpack_create(&cmt, (char *) data, len, &offset); | ||
| if (ret != CMT_DECODE_MSGPACK_SUCCESS) { | ||
| flb_error("[proxy] failed to decode metrics msgpack (error %d)", ret); | ||
| proxy_go_input_cleanup(ctx, data); | ||
| return -1; | ||
| } | ||
| ret = flb_input_metrics_append(ins, NULL, 0, cmt); | ||
| if (ret != 0) { | ||
| flb_error("[proxy] could not append metrics, ret=%d", ret); | ||
| cmt_decode_msgpack_destroy(cmt); | ||
| proxy_go_input_cleanup(ctx, data); | ||
| return -1; | ||
| } | ||
| cmt_decode_msgpack_destroy(cmt); | ||
| } | ||
| else if (event_type == FLB_INPUT_TRACES) { |
There was a problem hiding this comment.
For a proxy plugin that follows the existing proxy event_type convention used by outputs, metrics are registered as FLB_OUTPUT_METRICS (value 2) and traces as FLB_OUTPUT_TRACES (value 4); this new input path compares the same def->event_type against FLB_INPUT_METRICS/FLB_INPUT_TRACES (1/2). In that scenario a metrics-producing Go input is decoded as traces and a traces-producing input falls through to the log append path, so the feature fails for SDKs exposing the already-established proxy output constants unless this registration value is translated by plugin type.
Useful? React with 👍 / 👎.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/flb_plugin_proxy.c`:
- Around line 140-143: The current fallback in flb_plugin_proxy.c routes any
non-handled event_type into flb_input_log_append; update the branch so only
FLB_INPUT_LOGS is passed to flb_input_log_append and all other event_type values
(e.g., FLB_INPUT_BLOBS, FLB_INPUT_PROFILES) are rejected immediately: replace
the bare else with an explicit if (event_type == FLB_INPUT_LOGS) { ret =
flb_input_log_append(...); } else { set ret to an error value, emit an
explanatory error/log mentioning the unexpected event_type and the plugin/input
instance, and return/fail fast instead of treating it as logs } to preserve the
logs vs non-logs contract and avoid misrouting payloads.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
Read event_type from the proxy definition and dispatch to the appropriate append function (flb_input_metrics_append or flb_input_trace_append) in the collect callback, mirroring how the output proxy side already handles event_type. Metrics and traces are decoded from msgpack using cmt_decode_msgpack and ctr_decode_msgpack respectively before being appended. Logs remain the default (event_type == FLB_INPUT_LOGS) to preserve backwards compatibility. Unsupported event types are rejected with an error. Error handling mirrors the pattern in fw_prot.c:append_log: return values are checked, resources are freed on failure, and ctr ownership is not assumed on the traces success path since flb_input_trace_append takes ownership of the context. fluent#11914 Signed-off-by: seblaz <sebastian.blazquez96@gmail.com>
seblaz
force-pushed
the
feat/proxy-input-event-type
branch
from
59f97d1 to
0f86281
Compare
1 participant
Allow proxy input plugins (e.g. Go plugins via fluent-bit-go) to emit metrics and traces event types, not just logs.
The proxy input collect callback previously hardcoded
flb_input_log_append, making it impossible for proxy-based input plugins to feed the metrics or traces pipeline. This change readsevent_typefrom the proxy plugin definition and dispatches to the appropriate append function:FLB_INPUT_METRICS→ decode msgpack withcmt_decode_msgpack_create, append viaflb_input_metrics_appendFLB_INPUT_TRACES→ decode msgpack withctr_decode_msgpack_create, append viaflb_input_trace_appendflb_input_log_appendbehavior, fully backwards compatibleThe output proxy side already supports
event_typewith the same pattern; this brings input plugins to parity. Error handling mirrorsfw_prot.c:append_log.The
event_typefield already exists onflb_plugin_proxy_def(unused for inputs until now), so Go plugins can set it duringFLBPluginRegisterwithout any SDK-side C changes — though the fluent-bit-go SDK will need to expose the field to users.Fixes #11914
Enter
[N/A]in the box, if an item is not applicable to your change.Testing
Before we can approve your change; please submit the following in a comment:
If this is a change to packaging of containers or native binaries then please confirm it works for all targets.
ok-package-testlabel to test for all targets (requires maintainer to do).Documentation
Backporting
Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.
Summary by CodeRabbit
New Features
Bug Fixes