Skip to content

Commit 60185dd

Browse files
committed
GPU: wait for the O2 track count before sorting the O2 output tracks
With GPUCA_SPECIALIZE_THRUST_SORTS (CUDA and HIP), the host-side launch of GPUTPCGMO2Output::sort takes the number of elements from the host copy of the merger's memory, processors()->tpcMerger.NOutputTracksTPCO2(). That count is computed by GPUTPCGMO2Output::prepare on the device, and TransferMemoryResourceLinkToHost brings it to the host asynchronously, on the same stream, right before the sort is launched. The event of the transfer was only waited for after the sort, so the sort very likely got the value from before prepare ran: 0, since RunTPCTrackingMerger clears the merger's memory on the host at its start, and the transfer is queued behind prepare. Then the O2 output tracks keep the order in which prepare wrote them. This goes back to b0155c2, which moved the wait after the sort. With debugLevel >= 1 or serializeGPU the transfer is synchronous, which would hide it. The other sorts that read host counts, SortTracks and SortTracksQPt, wait for their transfer with CondWaitEvent first; do the same here, by waiting before the sort. The wait was already needed before the allocation of the O2 output that follows, so it costs no extra synchronisation. Found by reading the code, while porting these sorts to the Metal backend, which reads the count after draining the stream. Not tested on CUDA or HIP, there is no NVIDIA or AMD GPU here; the CPU build compiles. To check on a GPU: with debugLevel 0, outputTracksTPCO2 should come out ordered by the sort's comparator with this change, and not without it.
1 parent 89eeccd commit 60185dd

1 file changed

Lines changed: 1 addition & 1 deletion

File tree

‎GPU/GPUTracking/Global/GPUChainTrackingMerger.cxx‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -305,9 +305,9 @@ int32_t GPUChainTracking::RunTPCTrackingMerger(bool synchronizeOutput)
305305
runKernel<GPUTPCGMO2Output, GPUTPCGMO2Output::prepare>(GetGridAuto(0, deviceType));
306306
}
307307
TransferMemoryResourceLinkToHost(RecoStep::TPCMerging, Merger.MemoryResMemory(), 0, &mEvents->single);
308+
SynchronizeEventAndRelease(mEvents->single, doGPU); // The CUDA / HIP specialization of the sort reads nO2Tracks on the host
308309
runKernel<GPUTPCGMO2Output, GPUTPCGMO2Output::sort>(GetGridAuto(0, deviceType));
309310
mRec->ReturnVolatileDeviceMemory();
310-
SynchronizeEventAndRelease(mEvents->single, doGPU);
311311

312312
if (GetProcessingSettings().clearO2OutputFromGPU) {
313313
mRec->MakeFutureDeviceMemoryAllocationsVolatile();

0 commit comments

Comments
 (0)