feat: add batch_size support to opentelemetry output plugin

[mvitz]

What this PR does / why we need it:

It adds the batch_size option for the OpenTelemetry output plugin.

Does this PR introduced a user-facing change?

Add batchSize int32 option to opentelemetry output plugin.

Additional documentation, usage docs, etc.:

[Copilot]

Pull request overview

Note

Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.

Adds batchSize configuration support for the Fluent Bit OpenTelemetry output plugin across the API type, generated CRDs, Helm chart CRDs, and user documentation.

Changes:

• Introduces batchSize (*int32) to the OpenTelemetry output plugin API and maps it to Fluent Bit’s batch_size key.
• Updates CRD schemas (bases, rendered manifests, and chart CRDs) to include the new field.
• Updates plugin documentation and extends unit tests to cover the new parameter.

Reviewed changes

Copilot reviewed 10 out of 11 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
manifests/setup/setup.yaml	Adds batchSize to rendered CRD manifests so clusters can validate the new field.
docs/plugins/fluentbit/output/open_telemetry.md	Documents the new batchSize option in the OpenTelemetry output plugin page.
config/crd/bases/fluentbit.fluent.io_outputs.yaml	Adds batchSize schema to the Output CRD base.
config/crd/bases/fluentbit.fluent.io_clusteroutputs.yaml	Adds batchSize schema to the ClusterOutput CRD base.
charts/fluent-operator/crds/fluentbit.fluent.io_outputs.yaml	Propagates batchSize schema into the Helm chart CRDs for Outputs.
charts/fluent-operator/crds/fluentbit.fluent.io_clusteroutputs.yaml	Propagates batchSize schema into the Helm chart CRDs for ClusterOutputs.
charts/fluent-operator-fluent-bit-crds/templates/fluentbit.fluent.io_outputs.yaml	Updates the fluent-bit CRD chart template schema for Outputs.
charts/fluent-operator-fluent-bit-crds/templates/fluentbit.fluent.io_clusteroutputs.yaml	Updates the fluent-bit CRD chart template schema for ClusterOutputs.
apis/fluentbit/v1alpha2/plugins/output/open_telemetry_types_test.go	Extends unit test expectations to include batch_size.
apis/fluentbit/v1alpha2/plugins/output/open_telemetry_types.go	Adds BatchSize field and inserts batch_size into generated Fluent Bit config KVs.

Files not reviewed (1)

• apis/fluentbit/v1alpha2/plugins/output/zz_generated.deepcopy.go: Generated file

Comments suppressed due to low confidence (1)

manifests/setup/setup.yaml:1

• This field is added in several generated CRD artifacts (bases, rendered manifests, and multiple Helm chart CRDs/templates). To reduce drift risk, prefer making the change in the single source of truth (Go type + controller-gen markers) and regenerating all CRD outputs, rather than maintaining the same schema snippet across multiple files.

[joshuabaird]

@mvitz Thanks for the contribution. Couple things:

The Basic test and verify CI job is failing on the "Ensure all generated code is committed" step because docs/plugins/fluentbit/output/open_telemetry.md was hand-edited to add , default 1000``, but that doc is auto-generated from the Go struct comment.

The doc generator (go run ./cmd/doc-gen/main.go) regenerates the row from the BatchSize field comment in apis/fluentbit/v1alpha2/plugins/output/open_telemetry_types.go, which does not contain the default text, so regeneration produces a diff and the check fails.

To fix, add the default to the Go struct comment so it propagates consistently:

// Set the maximum number of log records to be flushed at a time, default `1000`
BatchSize *int32 `json:"batchSize,omitempty"`

Then run the generate target (e.g. make manifests docs) and commit the regenerated files. Alternatively, drop , default 1000`` from the doc line so it matches the generated output.

[mvitz]

@joshuabaird thanks for the help. I dropped the mentioned default value from the docs because it's the implicit default of fluentbit itself and therefore should be clear without mentioning it within the operators doc.
I also added the suggestet validation from copilot and set it to a minimum of zero as this allows to force using the default value of fluentbit.
Everything was force pushed because I don't think a separate commit would have given any value.