Skip to content

config_format: yaml: Handle list format of parsers_file on service section#11926

Open
cosmo0920 wants to merge 3 commits into
masterfrom
cosmo0920-handle-multi-parsers_file-on-service-section
Open

config_format: yaml: Handle list format of parsers_file on service section#11926
cosmo0920 wants to merge 3 commits into
masterfrom
cosmo0920-handle-multi-parsers_file-on-service-section

Conversation

@cosmo0920

@cosmo0920 cosmo0920 commented Jun 8, 2026

Copy link
Copy Markdown
Contributor

In this PR, we support a list format of parsers_file.

Closes #11910.


Enter [N/A] in the box, if an item is not applicable to your change.

Testing
Before we can approve your change; please submit the following in a comment:

  • Example configuration file for the change
  • Debug log output from testing the change
  • Attached Valgrind output that shows no leaks or memory corruption was found

If this is a change to packaging of containers or native binaries then please confirm it works for all targets.

  • Run local packaging test showing all targets (including any new ones) build.
  • Set ok-package-test label to test for all targets (requires maintainer to do).

Documentation

  • Documentation required for this feature

Backporting

  • Backport to latest stable release.

Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.

Summary by CodeRabbit

  • New Features

    • YAML config now accepts a list of parser files in the service section and recognizes camelCase variants of the key, allowing multiple parser files to be specified.
  • Tests

    • Added fixtures and tests verifying multiple parser files are loaded from YAML lists (snake_case and camelCase) and included an extra test parser to exercise this behavior.

@coderabbitai

coderabbitai Bot commented Jun 8, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 1b2c6c1a-c283-4468-bf3e-ddf0477f0e6c

📥 Commits

Reviewing files that changed from the base of the PR and between 18995b7 and 670ede3.

📒 Files selected for processing (1)
  • src/config_format/flb_cf_yaml.c
🚧 Files skipped from review as they are similar to previous changes (1)
  • src/config_format/flb_cf_yaml.c

📝 Walkthrough

Walkthrough

The PR adds YAML sequence support for the parsers_file key in the service section by introducing a new parser state that collects scalar list elements as repeated properties; tests and fixtures validate loading two parser files from the YAML list.

Changes

Parsers file sequence list support

Layer / File(s) Summary
YAML state machine for sequence values
src/config_format/flb_cf_yaml.c
Added STATE_SECTION_VAL_LIST enum state and debug mapping, introduced a local normalized_key in consume_event(), extended STATE_SECTION_VAL to transition into the new list state when service.parsers_file is a YAML sequence, and implemented the list handler to append each scalar element into cf_section->properties and validate state on sequence end.
Test cases and fixtures for parsers list
tests/internal/config_format_yaml.c, tests/internal/data/config_format/yaml/parsers/parsers-list.yaml, tests/internal/data/config_format/yaml/parsers/parsers-list-camel-case.yaml, tests/internal/data/config_format/yaml/parsers/parsers-extra.conf
Added fixture macros FLB_PARSERS_LIST and FLB_PARSERS_LIST_CAMEL_CASE, a shared helper test_parser_conf_list_file(path) that loads the YAML and asserts two additional parsers are registered, two wrapper tests for normal and camel-case keys, registered both tests in TEST_LIST, and added parser fixtures referenced by the YAML lists.

🎯 3 (Moderate) | ⏱️ ~25 minutes

🐰 A list of parsers, how neat!
The YAML state machine's heartbeat,
Sequences now flow with grace,
Multiple configs find their place,
Fluent-bit's config is complete! 🌊

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'config_format: yaml: Handle list format of parsers_file on service section' accurately summarizes the main change: adding support for list format of the parsers_file property in YAML service section.
Linked Issues check ✅ Passed The pull request fulfills the primary objective from issue #11910: implementing support for YAML sequences (list format) for the service.parsers_file property to match classic conf behavior.
Out of Scope Changes check ✅ Passed All changes are within scope—parser state handling, test fixtures, test cases, and configuration files are all necessary to implement list format support for parsers_file.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch cosmo0920-handle-multi-parsers_file-on-service-section

Warning

There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure.

🔧 Infer (1.2.0)
src/config_format/flb_cf_yaml.c

src/config_format/flb_cf_yaml.c:20:10: fatal error: 'fluent-bit/flb_info.h' file not found
20 | #include <fluent-bit/flb_info.h>
| ^~~~~~~~~~~~~~~~~~~~~~~
1 error generated.
Error: the following clang command did not run successfully:
/opt/infer-linux-x86_64-v1.2.0/lib/infer/facebook-clang-plugins/clang/install/bin/clang-18
@/tmp/coderabbit-infer/670ede3960c93a315a8819802c92ef9ec8a263c8-fac6a9507f5ea77a/tmp/clang_command_.tmp.e40bbf.txt
++Contents of '/tmp/coderabbit-infer/670ede3960c93a315a8819802c92ef9ec8a263c8-fac6a9507f5ea77a/tmp/clang_command_.tmp.e40bbf.txt':
"-cc1" "-load"
"/opt/infer-linux-x86_64-v1.2.0/lib/infer/infer/bin/../../facebook-clang-plugins/libtooling/build/FacebookClangPlugin.dylib"
"-add-plugin" "BiniouASTExporter" "-plugin-arg-BiniouASTExporter" "-"
"-plugin-arg-BiniouASTExporter" "PREPEND_CURRENT_DIR=1"
"-plugin-arg-BiniouASTExporter" "MAX_STRING_SIZE=65535" "-cc1" "-triple"
"x86_64-unknown-linux-gnu" "-emit-obj

... [truncated 744 characters] ...

nux-x86_64-v1.2.0/lib/infer/facebook-clang-plugins/clang/install/lib/clang/18/include"
"-internal-isystem" "/usr/local/include" "-internal-isystem"
"/usr/lib/gcc/x86_64-linux-gnu/12/../../../../x86_64-linux-gnu/include"
"-internal-externc-isystem" "/usr/include/x86_64-linux-gnu"
"-internal-externc-isystem" "/include" "-internal-externc-isystem"
"/usr/include" "-Wno-ignored-optimization-argument" "-Wno-everything"
"-ferror-limit" "19" "-fgnuc-version=4.2.1" "-fskip-odr-check-in-gmf"
"-D__GCC_HAVE_DWARF2_CFI_ASM=1" "-o"
"/tmp/coderabbit-infer/fac6a9507f5ea77a/file.o" "-x" "c"
"src/config_format/flb_cf_yaml.c" "-O0" "-fno-builtin" "-include"
"/opt/infer-linux-x86_64-v1.2.0/lib/infer/infer/bin/../lib/clang_wrappers/global_defines.h"
"-Wno-everything"


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d6700422c3

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/config_format/flb_cf_yaml.c Outdated

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
tests/internal/config_format_yaml.c (1)

326-360: ⚡ Quick win

Add a negative test for malformed service.parsers_file lists.

This new test covers the success path only. Please add one failure-path case (e.g., non-scalar list item) and assert parse/load fails, so regressions in STATE_SECTION_VAL_LIST validation are caught.

Based on learnings: tests for behavior changes should include focused coverage and validate both success and failure paths for parser-related handling.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/internal/config_format_yaml.c` around lines 326 - 360, Add a negative
test alongside test_parser_conf_list that creates a YAML CF via
flb_cf_yaml_create(…, FLB_PARSERS_LIST, …) but with a malformed
service.parsers_file list (e.g., include a non-scalar item such as a map or
sequence inside the list), then call flb_config_load_config_format(config, cf)
and assert it fails (ret != 0) and that parsers count does not increase; use the
same setup/teardown (flb_config_init, config->conf_path, flb_cf_destroy,
flb_config_exit) and the same test helpers (TEST_CHECK, TEST_MSG) to validate
the failure-path for STATE_SECTION_VAL_LIST validation.

Source: Learnings

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/config_format/flb_cf_yaml.c`:
- Around line 2001-2004: The call to flb_cf_section_property_add (used with
conf, state->cf_section->properties, state->key, value) is pointer-returning
here but the code tests for "< 0"; change the check to test for NULL instead,
e.g. verify the returned pointer is non-NULL and on NULL call flb_error("unable
to add property") and handle the failure path the same way the negative-check
previously did (stop parsing/return error). Update the conditional around
flb_cf_section_property_add accordingly so dropped properties are detected.

---

Nitpick comments:
In `@tests/internal/config_format_yaml.c`:
- Around line 326-360: Add a negative test alongside test_parser_conf_list that
creates a YAML CF via flb_cf_yaml_create(…, FLB_PARSERS_LIST, …) but with a
malformed service.parsers_file list (e.g., include a non-scalar item such as a
map or sequence inside the list), then call
flb_config_load_config_format(config, cf) and assert it fails (ret != 0) and
that parsers count does not increase; use the same setup/teardown
(flb_config_init, config->conf_path, flb_cf_destroy, flb_config_exit) and the
same test helpers (TEST_CHECK, TEST_MSG) to validate the failure-path for
STATE_SECTION_VAL_LIST validation.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 9d4e8741-a21b-4093-90a2-6c34a6940f61

📥 Commits

Reviewing files that changed from the base of the PR and between 5880717 and d670042.

📒 Files selected for processing (4)
  • src/config_format/flb_cf_yaml.c
  • tests/internal/config_format_yaml.c
  • tests/internal/data/config_format/yaml/parsers/parsers-extra.conf
  • tests/internal/data/config_format/yaml/parsers/parsers-list.yaml

Comment thread src/config_format/flb_cf_yaml.c
cosmo0920 added 2 commits June 8, 2026 14:54
Signed-off-by: Hiroshi Hatake <hiroshi@chronosphere.io>
Signed-off-by: Hiroshi Hatake <hiroshi@chronosphere.io>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
tests/internal/config_format_yaml.c (1)

327-327: ⚡ Quick win

Use const char * for immutable fixture paths.

path is only read and all call sites pass string literals. Making it const char * tightens the helper contract and avoids accidental mutation later.

Suggested diff
-static void test_parser_conf_list_file(char *path)
+static void test_parser_conf_list_file(const char *path)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/internal/config_format_yaml.c` at line 327, The helper function
signature test_parser_conf_list_file currently takes a mutable char *path even
though it only reads the value and all callers pass string literals; change the
parameter type to const char *path to express immutability and prevent
accidental modification. Update the function declaration/definition for
test_parser_conf_list_file and any prototypes/usages to accept const char * so
callers remain compatible with string literals.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@tests/internal/config_format_yaml.c`:
- Line 327: The helper function signature test_parser_conf_list_file currently
takes a mutable char *path even though it only reads the value and all callers
pass string literals; change the parameter type to const char *path to express
immutability and prevent accidental modification. Update the function
declaration/definition for test_parser_conf_list_file and any prototypes/usages
to accept const char * so callers remain compatible with string literals.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: a8169978-7ae8-4361-9d59-e04edb0f3b8f

📥 Commits

Reviewing files that changed from the base of the PR and between d670042 and 18995b7.

📒 Files selected for processing (5)
  • src/config_format/flb_cf_yaml.c
  • tests/internal/config_format_yaml.c
  • tests/internal/data/config_format/yaml/parsers/parsers-extra.conf
  • tests/internal/data/config_format/yaml/parsers/parsers-list-camel-case.yaml
  • tests/internal/data/config_format/yaml/parsers/parsers-list.yaml
✅ Files skipped from review due to trivial changes (3)
  • tests/internal/data/config_format/yaml/parsers/parsers-list.yaml
  • tests/internal/data/config_format/yaml/parsers/parsers-list-camel-case.yaml
  • tests/internal/data/config_format/yaml/parsers/parsers-extra.conf
🚧 Files skipped from review as they are similar to previous changes (1)
  • src/config_format/flb_cf_yaml.c

Signed-off-by: Hiroshi Hatake <hiroshi@chronosphere.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support list format for parsers_file in YAML service section

1 participant