[python] Add multi-threaded prefetch for pytorch streaming read#7143

Merged

JingsongLi merged 11 commits intoapache:masterfrom

XiaoHongbo-Hope:import_multi_thread_into_pytorch

Feb 27, 2026

Commits on Jan 28, 2026

support multithread when to torch dataset
xiaohongbo
committed
add test case for prefetch_concurrency
xiaohongbo
committed
add comment back
xiaohongbo
committed
revert
xiaohongbo
committed
clean code
xiaohongbo
committed
clean code
xiaohongbo
committed
clean code
xiaohongbo
committed
clean code
xiaohongbo
committed
fix hang producer threads indefinitely issue
xiaohongbo
committed
fix code format
xiaohongbo
committed

Commits on Jan 31, 2026

add doc for prefetch_concurrency
xiaohongbo
committed