Skip to content

Improved aio handling in SubtocPreloader#249

Open
Ozaq wants to merge 1 commit intodevelopfrom
fix/fdb-651
Open

Improved aio handling in SubtocPreloader#249
Ozaq wants to merge 1 commit intodevelopfrom
fix/fdb-651

Conversation

@Ozaq
Copy link
Copy Markdown
Member

@Ozaq Ozaq commented Mar 13, 2026

Description

Improved aio handling in SubtocPreloader

Now suspends for up to 1 sec, then logs stall and retries.

Also addresses missing error handling and cancellation on exception
throwing.

Contributor Declaration

By opening this pull request, I affirm the following:

  • All authors agree to the Contributor License Agreement.
  • The code follows the project's coding standards.
  • I have performed self-review and added comments where needed.
  • I have added or updated tests to verify that my changes are effective and functional.
  • I have run all existing tests and confirmed they pass.

🌈🌦️📖🚧 Documentation FDB 🚧📖🌦️🌈
https://sites.ecmwf.int/docs/dev-section/fdb/pull-requests/PR-249

@Ozaq Ozaq changed the title Impprove logging when preloading SubTocs Improve logging when preloading SubTocs Mar 13, 2026
@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented Mar 13, 2026

Codecov Report

❌ Patch coverage is 55.10204% with 22 lines in your changes missing coverage. Please review.
✅ Project coverage is 71.14%. Comparing base (ef36e0f) to head (6401089).

Files with missing lines Patch % Lines
src/fdb5/toc/TocHandler.cc 55.10% 22 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##           develop     #249      +/-   ##
===========================================
- Coverage    71.22%   71.14%   -0.08%     
===========================================
  Files          376      376              
  Lines        23762    23787      +25     
  Branches      2478     2484       +6     
===========================================
- Hits         16924    16923       -1     
- Misses        6838     6864      +26     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@Ozaq Ozaq force-pushed the fix/fdb-651 branch 2 times, most recently from ad63eeb to 84d7851 Compare March 17, 2026 14:50
@Ozaq Ozaq marked this pull request as ready for review March 17, 2026 14:50
@Ozaq Ozaq requested review from Copilot, simondsmart and tbkr March 17, 2026 14:50
@Ozaq
Copy link
Copy Markdown
Member Author

Ozaq commented Mar 17, 2026

@claude[agent] Review

@Claude
Copy link
Copy Markdown

Claude AI commented Mar 17, 2026

@Ozaq I've opened a new pull request, #252, to work on those changes. Once the pull request is ready, I'll request review from you.

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the SubToc preloading path in TocHandler to improve robustness around AIO reads by adding timeout-based aio_suspend() handling, additional debug logging, and cleanup-on-exception behavior.

Changes:

  • Add a 1s timeout to aio_suspend() and log when reads appear stalled before retrying.
  • Add RAII cancellation/reaping for outstanding AIO requests during stack unwinding.
  • Update Python test CMake environments to further isolate against globally installed libraries via FINDLIBS_DISABLE_PACKAGE.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.

File Description
src/fdb5/toc/TocHandler.cc Adds AIO timeout/retry logging and introduces cleanup logic for AIO requests during errors.
tests/pyfdb/CMakeLists.txt Extends test environment variables to disable package-based library discovery.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@Ozaq Ozaq force-pushed the fix/fdb-651 branch 3 times, most recently from 95fdcd2 to 61ae463 Compare March 17, 2026 15:15
@Ozaq Ozaq assigned Ozaq and Claude Mar 17, 2026
@Ozaq
Copy link
Copy Markdown
Member Author

Ozaq commented Mar 17, 2026

@claude review

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR improves robustness/observability of asynchronous SubToc preloading by adding timeout-based stall logging and attempting to ensure AIO requests are cancelled/cleaned up on exceptions, and it adjusts Python test environments to avoid accidentally picking up globally installed dependencies.

Changes:

  • Add a 1s aio_suspend() timeout loop that logs stalled SubToc reads and retries.
  • Add RAII cleanup for file descriptors and AIO requests during stack unwinding.
  • Update tests/pyfdb CMake test environment to disable package-based findlibs resolution.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File Description
tests/pyfdb/CMakeLists.txt Adds FINDLIBS_DISABLE_PACKAGE=yes to test environments to avoid using globally installed packages/libs.
src/fdb5/toc/TocHandler.cc Enhances SubToc AIO preload logic with timeout logging and introduces RAII cancellation/reaping on exceptions.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@Ozaq Ozaq changed the title Improve logging when preloading SubTocs Improved aio handling in SubtocPreloader Mar 18, 2026
Now suspends for up to 1 sec, then logs stall and retries.

Also addresses missing error handling and cancellation on exception
throwing.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants