Add optional FlatBuffers messages and code generation

[AnDiXL]

What problem does this PR solve?

Related discussion: #2354.

This is the message/codegen follow-up to #3196, opened at @wwbmmm's request. It submits the two commits already shared there as a focused, independently reviewable PR.

Background and related work:

• Add SingleIOBuf for efficient flatbuffers serialization/deserialization #3062 (merged): the SingleIOBuf foundation used by this implementation.
• flatbuffers support(2/3): Add flatbuffers message construction logic #3196: the original FlatBuffers message-construction proposal and review discussion.
• flatbuffers support(3/3): Add flatbuffers protocol support for brpc #3197: the associated RPC proposal. Network transport and Channel/Controller/Server integration remain separate and are not included in this PR.
• LinQuickDev/brpc#45: related FlatBuffers integration and validation work by @Spicy-cream.

Thanks to @Q1ngbo for the original work, and to @Spicy-cream for the continued interest in FlatBuffers support and collaboration around these follow-ups.

What is changed and the side effects?

Changed:

• 3f2550c6: add IOBuf-backed Message, allocator-aware MessageBuilder, stable service/method descriptors, optional CMake/Make/Bazel integration, documentation and regression tests.
• 6e5a3b32: fix generated header-guard collisions and relocation, and make standalone codegen acceptance use the appropriate Protobuf includes, link target and C++ standard.
• Message derives from NonreflectableMessage<Message>; existing protobuf-facing Protocol callback signatures remain unchanged.
• The standalone brpc_flatc binding generator uses the upstream FlatBuffers parser rather than requiring a compiler fork.

Scope and side effects:

• FlatBuffers support is disabled by default.
• This is message construction and generated in-process dispatch, not an fb_rpc transport or a network RPC example. The RPC follow-up will carry those separately.
• No measured performance improvement is claimed.

Validation

Recorded checks, not rerun for PR creation:

• Exact PR head (6e5a3b32), macOS/CMake: 21 message tests and both flatbuffers_codegen_acceptance and flatbuffers_codegen_runtime passed, with no failures or skips.
• Related full message/RPC snapshots were also validated on Linux with CMake, Make and Bazel. This is integration evidence, not a separate Linux validation of this reduced two-commit PR.
• Full-repository tests, sanitizer/performance results and hosted CI for this new PR are not claimed here.

Build and usage instructions are in docs/en/flatbuffers.md and tools/flatbuffers/README.md.

---

Check List:

• Focused build and test validation is recorded above.
• Message and code-generation regression tests are included.
• Build instructions and ownership/verification limitations are documented.
• Hosted CI for this PR has completed.

[Copilot]

Copilot review overview

🔵 Needs a closer look

The ownership-sensitive allocator implementation, generated public API, and three build-system integrations warrant final human review.

Review effort: Balanced
Findings: None

What changed in this PR

Adds optional FlatBuffers-backed messages, service descriptors, and standalone service-binding generation without introducing network transport.

Changes:

• Implements move-only messages, builders, allocators, and descriptors.
• Adds brpc_flatc generation and documentation.
• Integrates optional support and tests across Make, CMake, and Bazel.

File	Description
WORKSPACE	Pins FlatBuffers for Bazel.
MODULE.bazel	Adds the Bzlmod FlatBuffers archive.
BUILD.bazel	Adds conditional sources and dependency.
CMakeLists.txt	Adds optional CMake support.
Makefile	Includes FlatBuffers sources conditionally.
config.h.in	Defines the feature macro.
config_brpc.sh	Adds Make configuration support.
docs/​en/​flatbuffers.md	Documents construction and descriptor APIs.
src/​brpc/​flatbuffers/​message.h	Declares messages, builders, and allocator.
src/​brpc/​flatbuffers/​message.cpp	Implements storage and serialization behavior.
src/​brpc/​flatbuffers/​service.h	Declares service descriptor interfaces.
src/​brpc/​flatbuffers/​service.cpp	Implements descriptor validation and lookup.
tools/​flatbuffers/​CMakeLists.txt	Builds the standalone generator.
tools/​flatbuffers/​README.md	Documents generator usage and contracts.
tools/​flatbuffers/​brpc_flatc.cpp	Generates service bindings from schemas.
test/​Makefile	Adds conditional Make tests.
test/​CMakeLists.txt	Adds FlatBuffers CMake tests.
test/​BUILD.bazel	Adds conditional Bazel tests.
test/​flatbuffers_message.fbs	Defines message test data.
test/​brpc_flatbuffers_unittest.cpp	Tests messages, allocators, and descriptors.
test/​flatbuffers_codegen/​echo.fbs	Defines code-generation fixtures.
test/​flatbuffers_codegen/​CMakeLists.txt	Configures code-generation tests.
test/​flatbuffers_codegen/​acceptance.cmake	Tests generated-code acceptance.
test/​flatbuffers_codegen/​runtime.cpp	Tests generated dispatch behavior.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[wwbmmm]

Could you also add a chinese version of this document?

[AnDiXL]

Could you also add a chinese version of this document?

Of course, I will submit it tomorrow.

[AnDiXL]

Added docs/cn/flatbuffers.md and linked it from the English document. This Chinese version is scoped to the message/codegen part of this PR; the RPC/example documentation remains in the follow-up draft.

[wwbmmm]

I think this PR is more comprehensive than #3196 . @Q1ngbo What do you think of this PR? Does it conflict with your previous work on flatbuffer?

[chenBright]

What is different about bazel_dep(name = "flatbuffers", version = "25.2.10")?

[AnDiXL]

bazel_dep(name = "flatbuffers", version = "25.2.10") is not equivalent here. The BCR module for FlatBuffers 25.2.10 imports gRPC and several JS/Go/Swift tooling dependencies for the full upstream build, while this PR only needs //:runtime_cc and //:flatc. Pulling that module would also bring gRPC's dependency graph, including another BoringSSL version, into Bzlmod resolution even when BRPC_WITH_FLATBUFFERS is false.

I kept the checksum-pinned http_archive and expanded the comment in MODULE.bazel to make this explicit.

[Copilot]

Copilot review overview

🔵 Needs a closer look

Make-generated headers can shadow CMake outputs, and hosted CI currently skips all FlatBuffers test paths.

Review effort: Balanced
Findings: None

Previously missed (2)

In code that hasn't changed since last review

Generate Make header outside source tree to prevent stale shadowing

test/​Makefile:245

Generating this header directly in test/ contaminates the other build paths. Because brpc_flatbuffers_unittest.cpp uses a quoted include, a header left by Make is searched before CMake's ${CMAKE_CURRENT_BINARY_DIR} output, so a later CMake build can silently compile a stale header (or fail its FlatBuffers version assertion); the file is also not covered by the generated-file patterns in .gitignore. Generate it in an ignored Make build directory and add that directory to this target's include path so the source tree can never shadow CMake's generated output.

Enable FlatBuffers tests and codegen coverage in CI

test/​BUILD.bazel:260

This test is incompatible unless --define=BRPC_WITH_FLATBUFFERS=true is passed, but none of the current GitHub Actions Bazel, CMake, or Make jobs enables FlatBuffers (and no job invokes the standalone codegen suite). Consequently all new runtime and generator behavior is skipped by hosted CI despite the repository's otherwise comprehensive C++ test runs. Add an enabled CI job that installs matching FlatBuffers headers/flatc and runs both this unit test and the standalone codegen acceptance/runtime tests.

[AnDiXL]

For the hosted FlatBuffers ON matrix: I intentionally kept it in the follow-up draft #3559 rather than this message/codegen PR. #3559 adds .github/workflows/flatbuffers-on.yml plus the validation runner and covers the FlatBuffers-enabled CMake/Make/Bazel paths together with the RPC/example changes. This PR keeps the smaller message/codegen scope.

[wwbmmm]

[replied by brpc-oncall robot] I agree with Copilot's two findings (the Make-generated header can shadow CMake's output, and hosted CI does not exercise any of the new FlatBuffers tests). My own remaining concerns are: the runtime binds to FlatBuffers internals without any version validation, Message::MergeFrom aborts the process rather than reporting an error, and the receive path's size/verification obligations are not stated. Please also confirm the service.h dispatch contract is what the RPC transport PR will consume, since it becomes public API here.

[wwbmmm]

[replied by brpc-oncall robot] WITH_FLATBUFFERS is validated only by the existence of flatbuffers/flatbuffers.h, but the runtime does not use FlatBuffers purely through its public API: src/brpc/flatbuffers/message.cpp reaches into builder internals (buf_, buf_.swap_allocator(), scratch_push_small(), string_pool, minalign_), and SlabAllocator asserts on the exact allocate()/reallocate_downward() size bookkeeping (only one live allocation, old_size == _capacity). Bazel pins 25.2.10 while CMake and config_brpc.sh accept any installed version, so with a different FlatBuffers an unsupported version fails deep in compilation or aborts inside the allocator. Please add a compile-time guard in src/brpc/flatbuffers/message.h (e.g. #if !defined(FLATBUFFERS_VERSION_MAJOR) || FLATBUFFERS_VERSION_MAJOR < X -> #error) and document the supported range, so users get a clear diagnostic instead of an obscure failure.

[wwbmmm]

[replied by brpc-oncall robot] parse_msg_from_iobuf (and therefore ParseFbFromIOBUF) accepts any msg_size below FLATBUFFERS_MAX_BUFFER_SIZE and, for fragmented or insufficiently aligned input, copies that many bytes into a fresh allocation before anything is verified. Once the RPC transport lands this length comes from an untrusted peer. Please state the caller contract here (and in docs/en/flatbuffers.md): the caller must bound msg_size with its own max-message-size setting and must call Verify<T>() before reading, so the receive-side limit is not left implicit.