Skip to content

Fix possible data race while calling has_ready_entities in EventsCBGExecutor - #3288

Open
armaho wants to merge 2 commits into
ros2:rollingfrom
armaho:data_race_events_cbg_executor
Open

armaho wants to merge 2 commits into
ros2:rollingfrom
armaho:data_race_events_cbg_executor

Conversation

@armaho

@armaho armaho commented Sep 28, 2026

Copy link
Copy Markdown
Contributor

Description

In #3178, I called has_ready_entities method of FirstInFirstOutCallbackGroupHandle without holding ready_mutex. This can result to data races while accessing ready_entities. This PR fixes that.

Is this user-facing behavior change?

Did you use Generative AI?

No

Additional Information

…xecutor

Signed-off-by: Arman Hosseini <armanhosseini878787@gmail.com>
@armaho
armaho marked this pull request as draft September 28, 2026 17:39
@armaho
armaho marked this pull request as ready for review September 28, 2026 17:45
@github-actions

github-actions Bot commented Sep 28, 2026 •

Copy link
Copy Markdown

ABI Compliance Check

✅ Verdict: compatible

Library Verdict Summary
libcomponent_manager.so ✅ compatible No ABI changes detected.
librclcpp.so ✅ compatible No ABI changes detected.
librclcpp_action.so ✅ compatible No ABI changes detected.
librclcpp_lifecycle.so ✅ compatible No ABI changes detected.
✅ libcomponent_manager.so — full abidiff report

Compared:

  • Base: lib-base/libcomponent_manager.so
  • Head: lib-pr/libcomponent_manager.so @ 67e878e
(empty report — no differences printed by abidiff)
✅ librclcpp.so — full abidiff report

Compared:

  • Base: lib-base/librclcpp.so
  • Head: lib-pr/librclcpp.so @ 67e878e
(empty report — no differences printed by abidiff)
✅ librclcpp_action.so — full abidiff report

Compared:

  • Base: lib-base/librclcpp_action.so
  • Head: lib-pr/librclcpp_action.so @ 67e878e
(empty report — no differences printed by abidiff)
✅ librclcpp_lifecycle.so — full abidiff report

Compared:

  • Base: lib-base/librclcpp_lifecycle.so
  • Head: lib-pr/librclcpp_lifecycle.so @ 67e878e
(empty report — no differences printed by abidiff)

Updated for commit 67e878e · check run · suppressions: /home/runner/work/_temp/ros2-abi-suppressions.txt

@fujitatomoya fujitatomoya left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm with green CI.

@jmachowinski can you take a look at it just in case?

@fujitatomoya

Copy link
Copy Markdown
Collaborator

Pulls: #3288
Gist: https://gist.githubusercontent.com/fujitatomoya/2e111bb8d843206ce0f30666e3ee13a2/raw/27a5a0a5eeaa38ccf3f8451b413d112403e7d2e1/ros2.repos
BUILD args: --packages-above-and-dependencies rclcpp
TEST args: --packages-above rclcpp
ROS Distro: rolling
Job: ci_launcher
ci_launcher ran: https://ci.ros2.org/job/ci_launcher/20563

  • Linux Build Status
  • Linux-aarch64 Build Status
  • Linux-rhel Build Status
  • Windows Build Status

@armaho

armaho commented Sep 29, 2026

Copy link
Copy Markdown
Contributor Author

@fujitatomoya

if approved, #3290 can also fix this and I think it’s a better approach.

@jmachowinski

Copy link
Copy Markdown
Collaborator

This PR is not valid.
@armaho in_queue is protected by the ready_callback_groups_mutex and it is always held whenever we modify the variable.
We should perhaps document this.

Taking the second mutex here is also harmful as it might introduce an lock order inversion and therefore a deadlock.

I'll close this PR as invalid.

@armaho

armaho commented Oct 2, 2026

Copy link
Copy Markdown
Contributor Author

@jmachowinski

I was worried about the ready_cbg->has_ready_entities() call without holding CallbackGroupHandle::ready_mutex, not the in_queue field. Since the add_ready_entity() method only holds ready_mutex while modifying the handle's internal queue.

@jmachowinski

Copy link
Copy Markdown
Collaborator

You are right, this is a problem....

@armaho

armaho commented Oct 2, 2026

Copy link
Copy Markdown
Contributor Author

@jmachowinski

Taking the second mutex here is also harmful as it might introduce an lock order inversion and therefore a deadlock.

Sorry I didn't quite understand this. Don't we take that second mutex anyways in the ready_cbg->get_next_ready_entity() call?

@jmachowinski

Copy link
Copy Markdown
Collaborator

You are right, we do, okay, in this case it should be fine to take it in this order.
Man... I wrote this code 2 years ago and forgot almost all details ;-)

@armaho

armaho commented Oct 2, 2026

Copy link
Copy Markdown
Contributor Author

Okay so this pr fixes the problem if #3290 doesn't get approved :)

@skyegalaxy

Copy link
Copy Markdown
Member

Pulls: #3288
Gist: https://gist.githubusercontent.com/skyegalaxy/4ebc1c4326f05ad20b6faeb404a77c9b/raw/19edc67ea80399799ab2dadd164761a9aa3c8d54/ros2.repos
BUILD args: --packages-above-and-dependencies rclcpp
TEST args: --packages-above rclcpp
ROS Distro: rolling
Job: ci_launcher
ci_launcher ran: https://ci.ros2.org/job/ci_launcher/20833

  • Linux Build Status
  • Linux-aarch64 Build Status
  • Linux-rhel Build Status
  • Windows Build Status

@skyegalaxy

Copy link
Copy Markdown
Member

@armaho , could you please rebase on latest rolling? Looks like the removed statistics collector stuff is breaking the build for this branch

@skyegalaxy

Copy link
Copy Markdown
Member

Pulls: #3288
Gist: https://gist.githubusercontent.com/skyegalaxy/2408fe7f3092d30955331403d11b39ea/raw/19edc67ea80399799ab2dadd164761a9aa3c8d54/ros2.repos
BUILD args: --packages-above-and-dependencies rclcpp
TEST args: --packages-above rclcpp
ROS Distro: rolling
Job: ci_launcher
ci_launcher ran: https://ci.ros2.org/job/ci_launcher/20839

  • Linux Build Status
  • Linux-aarch64 Build Status
  • Linux-rhel Build Status
  • Windows Build Status

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants