Skip to content

fix(event-dispatcher): synchronize dispatcher shutdown state - #3738

Open
songgaoye wants to merge 3 commits into
apache:masterfrom
songgaoye:fix/event-dispatcher-stop-race
Open

songgaoye wants to merge 3 commits into
apache:masterfrom
songgaoye:fix/event-dispatcher-stop-race

Conversation

@songgaoye

@songgaoye songgaoye commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

What problem does this PR solve?

Issue Number: resolve

Problem Summary:

EventDispatcher::_stop is read by Run() and Running() while Stop()
writes it from another thread. The existing volatile bool does not
provide atomicity, so these accesses form a C++ data race.

What is changed and the side effects?

Changed:

  • Replace _stop with butil::atomic<bool>.
  • Use relaxed loads and stores in both epoll and kqueue implementations.
    _stop only requests shutdown and does not publish other shared state.
  • Remove comments treating epoll/kqueue operations as memory fences.
  • Check that Running() returns false immediately after Stop(), before
    Join() clears the thread ID.
  • Add a TSAN-oriented regression test using a relaxed handoff between
    Stop() and Running().
  • Use relaxed ordering for the related test's completion flag;
    pthread_join() provides completion synchronization.

Side effects:

  • Performance effects:

    Shutdown-state accesses are atomic with relaxed ordering.

  • Breaking backward compatibility:

    None.

Check List:

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The regression test does not deterministically overlap the reader with Stop(), so it can miss the intended race and allow the prior unsafe implementation to pass.

1 open issue
What changed in this PR

This pull request fixes a data race in EventDispatcher shutdown handling by replacing the volatile stop flag with an atomic flag and applying release/acquire synchronization across shutdown, status checks, and epoll/kqueue event loops.

Changes:

  • Replaces _stop with butil::atomic<bool>.
  • Uses release stores and acquire loads for shutdown state.
  • Adds lifecycle and concurrent Running()/Stop() regression tests.
File Description
test/​brpc_event_dispatcher_unittest.cpp Adds dispatcher lifecycle and concurrent Running()/Stop() regression tests.
src/​brpc/​event_dispatcher.h Declares the atomic shutdown state and includes atomic support.
src/​brpc/​event_dispatcher_kqueue.cpp Synchronizes shutdown checks and state publication in the kqueue dispatcher.
src/​brpc/​event_dispatcher_epoll.cpp Synchronizes shutdown checks and state publication in the epoll dispatcher.

Review effort: Lite

Comment thread test/brpc_event_dispatcher_unittest.cpp Resolved

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The shutdown state now uses atomic acquire/release synchronization across event-loop paths, with regression tests covering lifecycle and concurrent shutdown behavior.

No open issues 🎉

1 resolved since last review

Review effort: Lite

Comment thread src/brpc/event_dispatcher_epoll.cpp Resolved
Comment thread test/brpc_event_dispatcher_unittest.cpp Resolved
@songgaoye

Copy link
Copy Markdown
Contributor Author
WARNING: ThreadSanitizer: data race (pid=318516)

  Write of size 1 at 0x7fffffffe3fc by main thread:
    #0 brpc::EventDispatcher::Stop()
       src/brpc/event_dispatcher_epoll.cpp:101
    #1 EventDispatcherTest_running_is_safe_after_stop_Test::TestBody()
       test/brpc_event_dispatcher_unittest.cpp:172
    ...

  Previous read of size 1 at 0x7fffffffe3fc by thread T10:
    #0 brpc::EventDispatcher::Run()
       src/brpc/event_dispatcher_epoll.cpp:204
    #1 brpc::EventDispatcher::RunThis(void*)
       src/brpc/event_dispatcher_epoll.cpp:199
    #2 bthread::TaskGroup::task_runner(long)
       src/bthread/task_group.cpp:367
    #3 bthread_make_fcontext
    #4 bthread::TaskGroup::sched_to(bthread::TaskGroup**, unsigned long)
       src/bthread/task_group_inl.h:82
    #5 bthread::TaskGroup::run_main_task()
       src/bthread/task_group.cpp:173
    #6 bthread::TaskControl::worker_thread(void*)
       src/bthread/task_control.cpp:135

  Location is stack of main thread.

SUMMARY: ThreadSanitizer: data race
src/brpc/event_dispatcher_epoll.cpp:101
in brpc::EventDispatcher::Stop()

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants