You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
[Bug] ScanContextBuilder spawns eight executor threads per builder even when an executor is supplied #287
Build a ScanContextBuilder per point lookup, as serving code does, and call Finish().
Trace thread creation, for example strace -f -c -e trace=clone around the lookups.
Each ScanContextBuilder costs eight clone calls: ScanContextBuilder::Impl calls CreateDefaultExecutor() in its member initializer and again in Reset(), which Finish() invokes right after building the context, and every DefaultExecutor spawns DEFAULT_EXECUTOR_THREAD_COUNT (4) worker threads in its constructor and joins them on destruction. This happens even when the caller supplies its own executor through WithExecutor(). Measured on a point-lookup path, about nine clone calls per lookup, eight of them from this builder.
What doesn't meet your expectations?
Constructing or resetting a builder should not create threads. A context that is given an executor through WithExecutor() should create none at all, and the default executor's workers should only exist once the scan actually submits work.
Anything else?
ReadContextBuilder already resolves its default executor lazily in Finish(); ScanContextBuilder should behave the same. Falling back to the process-wide GetGlobalDefaultExecutor() is not an option: it sizes its pool by CPU count and FileStoreScan::ReadFileEntries() blocks in CollectAll() on the same pool, which can deadlock once every worker is inside a scan.
Activity