Benchmark suite: PointCloudBVH vs picoflann/nanoflann, TriMeshSDF vs fcpw (#109) - #112
Closed
rmrsk wants to merge 17 commits into
Closed
Benchmark suite: PointCloudBVH vs picoflann/nanoflann, TriMeshSDF vs fcpw (#109)#112rmrsk wants to merge 17 commits into
rmrsk wants to merge 17 commits into
Conversation
Closed
5 tasks
…n, fcpw Toward a benchmark suite (issue #109), gather all submodules under a single Submodules/ directory: - Move common-3d-test-models -> Submodules/common-3d-test-models, updating every relative mesh path in the Examples (MeshSDF, CSGUnion) and Integrations (AMReX MeshSDF/PaintEB) main.cpp defaults, Examples/CMakeLists.txt, and the docs (ObtainingEBGeometry/ExampleMeshSDF/ExampleCSGUnion.rst, READMEs) + the REUSE.toml note. URLs and the submodule's own name are left untouched. Verified MeshSDF and CSGUnion still load the mesh at the new path. - Add the benchmark comparison libraries as submodules: Submodules/nanoflann, Submodules/picoflann, Submodules/fcpw (all shallow). fcpw's Eigen dependency is its own nested submodule (deps/eigen). reuse lint compliant (submodule contents carry their own licensing). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HS5LFQihKoMrE6nbPaQW9R
…F vs fcpw Per issue #109, add a benchmark suite comparing EBGeometry against other geometry-query libraries (pinned as submodules under Submodules/). Illustrative and not CI-tested, like Integrations/. - Benchmark/NearestNeighbor: all-nearest-neighbor over a 500k point cloud -- PointCloudBVH vs picoflann vs nanoflann. Flann queries iterated in Hilbert order for a fair (warm-cache) comparison; the one-time sort they need is reported separately (EBGeometry reuses its build order for free). PointCloudBVH ~0.29 us/pt, nanoflann ~0.25, picoflann ~0.47; folding the sort in, PointCloudBVH wins the end-to-end job. - Benchmark/MeshSDF: closest-point on a triangle mesh (armadillo, ~100k tris, float) -- TriMeshSDF vs fcpw, the task both are built for. Mesh parsed once; each builds its own BVH over the same triangles and answers the same queries; distances cross-checked (0 mismatches). TriMeshSDF ~2.6 us/query vs fcpw ~7.5 -- with the caveat, documented, that fcpw runs its scalar Eigen fallback here (Enoki off) while TriMeshSDF is SIMD, so fcpw's vectorized path would narrow the query gap. Adds a Benchmark.rst docs page (wired into the toctree) and a REUSE.toml block for the benchmark build files/READMEs (main.cpp carry inline SPDX). Both benchmarks build via their GNUmakefiles against the Submodules/ deps. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HS5LFQihKoMrE6nbPaQW9R
… to the comparison - fcpw is now built with its Enoki CPU vectorization (FCPW_USE_ENOKI, vectorized MBVH) instead of the scalar Eigen fallback -- a like-for-like float/SIMD comparison with TriMeshSDF. Enoki's headers are vendored inside the fcpw submodule, so no extra fetch is needed; FCPW_SIMD_WIDTH is a GNUmakefile variable (default 8 = AVX2; set 4 for SSE, 16 for AVX-512). - Add TriangleMeshDistance (https://github.com/InteractiveComputerGraphics/TriangleMeshDistance) as a third contender, pinned as a submodule under Submodules/. It is a header-only, double-precision, scalar signed-distance library -- a widely-used reference point, noted as not a same-precision comparison. Its distances are cross-checked against TriMeshSDF alongside fcpw's (0 mismatches). Representative (armadillo ~100k triangles, 100k queries, one machine): TriMeshSDF build ~63 ms query ~2.6 us fcpw (Enoki) build ~46 ms query ~3.5 us (was ~7.5 us scalar) TriangleMeshDistance build ~115 ms query ~10.5 us (double, scalar) Updates the Benchmark README and Benchmark.rst accordingly. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HS5LFQihKoMrE6nbPaQW9R
…ompile-only CI Move the comparison-library submodules (nanoflann, picoflann, fcpw, TriangleMeshDistance) out of Submodules/ to live directly under Benchmark/, and move common-3d-test-models back to the repository top level where it has always been. Update .gitmodules, REUSE.toml, all example/integration path references, and the benchmark docs to match. Soften the benchmark framing in Benchmark/README.md and Docs/Sphinx/source/Benchmark.rst: present the suite as reproducible, cross-checked methodology whose single-machine numbers are illustrative snapshots rather than a scoreboard. Add a compile-only Benchmark-Compile CI job that fetches each benchmark's comparison-library submodules and builds (but never runs or times) the two programs via their GNUmakefiles, as a smoke test against bit-rot. Document the new job and the softened framing in ContinuousIntegration.rst. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JhdRJydutVJqQjg2Fuee9Z
…nchmark Extend Benchmark/NearestNeighbor to compare EBGeometry's PointCloudHashGrid alongside PointCloudBVH, picoflann, and nanoflann, and run the whole comparison over two point distributions in the same program: the existing uniform unit-cube cloud and a new uniformly-sampled unit-sphere surface (a 2D manifold that is locally dense but globally hollow -- a harder case for a uniform grid). Every method is still cross-checked against the brute-force baseline on both cases. Update Benchmark/README.md's NearestNeighbor section to describe the two distributions and the four methods, and to record the build/query tradeoff the hash grid exposes. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JhdRJydutVJqQjg2Fuee9Z
Following the nanoflann traversal study: nanoflann prunes the far child with a (1+eps)-loosened bound for approximate search. Add the equivalent to PointCloudBVH as an optional, backward-compatible knob -- allNearestNeighbors gains an a_eps parameter (default 0 = exact, unchanged behavior) that is converted to a squared-distance prune-scale 1/(1+eps)^2 and threaded through query() into both the seeded scalar DFS and the pruneTraverse-based paths. A returned neighbor is then guaranteed within a factor (1+eps) of the true nearest; best.distanceSquared always remains the true distance to a real point. Add a correctness test asserting the (1+eps) guarantee (and that eps==0 reproduces the exact search), and an eps-sweep section to the NearestNeighbor benchmark reporting query time, speedup, %exact, and worst-case distance ratio on both point distributions. Finding: the speedup is modest (~1.1-1.35x even at eps=2) because the all-NN path already seeds each query's bound from the point's own leaf, so the bound starts near-tight and there is little marginal pruning left for eps to remove -- unlike the loose-initial-bound high-dimensional kNN case where eps pays off. The separately-considered no-sort idea needs no prototype: the seeded all-NN path already uses an unordered scalar DFS, so the sort in pruneTraverse never runs on this workload. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JhdRJydutVJqQjg2Fuee9Z
…ark it" This reverts commit dc03101.
…-line fit) DefaultBranchingRatio<float>() returned 16 on AVX-512F (the SIMD-register-filling value). But the flat node stores K child offsets, so a K=16 float node is 96 B (24 B bounding volume + 8 B leaf fields + 16x4 B child offsets) and straddles two 64-byte cache lines, whereas K=8 is exactly 64 B -- one line. Nearest-neighbor traversal is memory-latency-bound, and the cache-line-sized node measured ~10-13% faster on PointCloudBVH all-nearest-neighbor (both uniform and sphere-surface distributions) than the wider K=16 fan-out, so cap float at 8 to match double. Every other (ISA, precision) default already fit one cache line, so only the AVX-512F/float case changes. The K=16/float SIMD path in pruneTraverse is kept for callers who request K=16 explicitly; it is simply no longer the default. Leaf SoA width (TriangleSoA/PointSoA DefaultWidth, still 16 for float) is unchanged. Update the DefaultBranchingRatio docs, the pruneTraverse configuration note, the Parser/PointCloudBVH @tparam comments, and the ImplemBVH/Parsers .rst pages (default-K table cell and rules of thumb) to reflect the cache-driven cap. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JhdRJydutVJqQjg2Fuee9Z
Add a "Finding" section to Benchmark/README.md recording why BVH::DefaultBranchingRatio<float>() is capped at 8 on AVX-512: a K=8 float node is one 64-byte cache line (24 B BV + 8 B leaf fields + 8x4 B child offsets) vs two lines at K=16, and both traversals are memory-latency-bound, so the cache-line-sized node measured faster (~7-14% on NearestNeighbor, ~6-9% query plus ~8% build on MeshSDF) despite the narrower SIMD fan-out. Notes the mesh case gives up a real 512-bit-load advantage and still wins, and the corollary that global float alone does not help query (K=16 re-inflates the node). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JhdRJydutVJqQjg2Fuee9Z
allNearestNeighborsPacket(packetSize) walks a packet of spatially-adjacent queries (consecutive in build order) through the tree together: each node is loaded once for the whole packet, a per-node uint64 active mask tracks which members still need the subtree, and each member seeds from its own leaf and excludes its own point. Results are bit-identical to serial allNearestNeighbors (new unit-test section checks packet sizes 1/3/8/64 against the serial graph for both precisions); the idea was to expose memory-level parallelism on the otherwise serial, latency-exposed dependent node-load chain. Measured outcome: it does not help this workload -- a consistent ~3-17% slower than serial at every packet size, and still slower even on a 4M-point cloud whose working set spills the 260 MB L3. The reason is that allNearestNeighbors already processes queries in spatial (m_order) order, so consecutive queries traverse an overlapping, slowly-sliding window of nodes/leaves that stays cache-resident regardless of total cloud size. The batch ordering already hides the latency packets aim to hide, so packets only add bookkeeping (mask iteration, leaf re-checks, fatter stack entries). Kept as an experimental method (clearly marked): the technique could still pay off for a batch of unordered external query points, where serial gets no cross-query cache reuse. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JhdRJydutVJqQjg2Fuee9Z
…otype" This reverts commit dcad7e1.
…rk README" This reverts commit 410f689.
…F (cache-line fit)" This reverts commit 8fca504.
Add https://github.com/KaruroChori/kd3 (a header-only SoA/SIMD KD-tree for low-dimensional point kNN) as a Benchmark/kd3 submodule and a fifth method in the all-nearest-neighbor benchmark, alongside PointCloudBVH, PointCloudHashGrid, picoflann, and nanoflann. Each point's nearest other point is found via kd3's query_knn(k=2) in Hilbert order (skipping the query point itself), cross-checked against the same brute-force baseline as the others. kd3 requires C++23 (std::expected/std::span), so the whole NearestNeighbor benchmark is now built with -std=c++23 (EBGeometry/nanoflann/picoflann all compile unchanged under it). For a like-for-like comparison kd3 is run in double and single-threaded (compiled without -fopenmp); its distance_t is pinned to the benchmark's T rather than its float default. Its headline float/SIMD/OpenMP fast path is therefore not exercised here -- noted in the code and README so the numbers aren't misread. In this restricted mode kd3 posts the fastest build and a competitive query. Wire kd3 into the compile-only Benchmark-Compile CI job's submodule list, and document it in Benchmark/README.md and Docs/Sphinx/source/Benchmark.rst (library list, fetch command, results table, and the parity caveats). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JhdRJydutVJqQjg2Fuee9Z
…enMP The prior note wrongly implied kd3's "~2.2x vs nanoflann" query claim relied on an OpenMP-parallel build. It does not: in kd3's own benchmark both the kd3 and nanoflann query loops are plain serial loops (query_knn_inline is a noexcept inline call; the printed "parallelism: 32" is cosmetic), so the speedup is kd3's SoA/SIMD per-query kernel in float, single-threaded. OpenMP only parallelizes the build (a separate, disclosed claim). The real reason kd3 doesn't reach 2x in this benchmark is that it is run in double, halving its SIMD width -- not threading. Compiling without -fopenmp keeps the build comparison single-threaded and thus fairer than kd3's own OpenMP-parallel build numbers. Fix the wording in main.cpp, Benchmark/README.md, and Benchmark.rst accordingly. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JhdRJydutVJqQjg2Fuee9Z
Alongside the same-precision kd3 (double) row, run kd3 in its native float precision (kd3::limits<float,3,float>) as a best-case reference -- its intended SoA/SIMD fast path -- and label the two rows kd3 (double) / kd3 (float). This is a different-precision reference, not apples-to-apples with the double field, mirroring how TriangleMeshDistance is reported (double) against float methods in the MeshSDF benchmark. The float row uses a looser cross-check tolerance since its squared distance is computed in float against the double brute-force truth. On this machine kd3 (float) runs ~7-11% faster than kd3 (double); the gain is modest here because AVX-512 clocks down, damping the float SIMD-width advantage. Update Benchmark/README.md (table + note) and Benchmark.rst accordingly. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JhdRJydutVJqQjg2Fuee9Z
The disabled-assertions branch expanded to ((void)(cond)), which still *evaluates* cond -- relying on the optimiser to dead-code-eliminate it. That elimination is not guaranteed: any assertion condition the compiler can't prove side-effect-free (e.g. the std::isfinite() precondition checks in the SoA distance kernels, evaluated on the order of 20x per nearest-neighbor query) stayed in the generated code. Switch the disabled expansion to (static_cast<void>(sizeof((cond)))). sizeof is an unevaluated context, so cond is parsed -- keeping it syntax-checked and keeping assertion-only variables/parameters from tripping unused-entity warnings -- but is never executed, at any optimisation level. Disabled assertions now have exactly zero runtime cost and cannot have side effects. Verified: InstantiateAll (all classes, both precisions) compiles clean under -Wall -Wextra with assertions off; debug (assertions on) and release-test (assertions off) test suites both pass. An interleaved A/B of the NearestNeighbor benchmark shows PointCloudBVH's uniform-cube query ~4-5% faster with the expressions truly gone. Update ConfigurationOptions.rst to match. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JhdRJydutVJqQjg2Fuee9Z
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a reproducible
Benchmark/suite comparing EBGeometry against other open-source geometry-querylibraries, and consolidates all git submodules under a single
Submodules/directory.Closes #109. Supersedes #110 (auto-closed when its base branch
particle_soa_pr3was merged anddeleted); this branch has been rebased onto
mainand carries the same two commits.Background
Issue #109 proposed a reproducible benchmark suite comparing EBGeometry against other open-source
geometry-query libraries, and consolidating git submodules under a single
Submodules/directory.This PR implements that. It builds on
PointCloudBVHandTriMeshSDF, both now merged tomain(#106, #108).
Solution
Submodule consolidation. All git submodules now live under
Submodules/:common-3d-test-models→Submodules/common-3d-test-models, rewiring every relative meshpath (Examples MeshSDF/CSGUnion, Integrations AMReX MeshSDF/PaintEB,
Examples/CMakeLists.txt, thedocs, and the
REUSE.tomlnote). URLs and the submodule's own name are untouched; verified MeshSDFand CSGUnion still load the mesh at the new path.
Submodules/nanoflann,Submodules/picoflann,Submodules/fcpw(fcpw's Eigen dependency is its own nested submodule).Benchmark/suite (illustrative, not CI-tested, likeIntegrations/):Benchmark/NearestNeighbor— all-nearest-neighbor over a 500k unit-cube point cloud (double):PointCloudBVHvs picoflann vs nanoflann. The KD-tree queries iterate in Hilbert order so theirnode cache is as warm as
PointCloudBVH's leaf-order batch (natural order is ~2× slower — unfair);the one-time spatial sort they need for that order is reported separately, since
PointCloudBVHreuses the ordering its build already produced for free. Representative:
PointCloudBVH~0.30,nanoflann ~0.24, picoflann ~0.47 µs/pt; folding the sort into the KD-trees,
PointCloudBVHwins theend-to-end job and builds ~1.5× faster than nanoflann.
Benchmark/MeshSDF— closest-point on a triangle mesh (armadillo, ~100k triangles):TriMeshSDFvs fcpw vsTriangleMeshDistance, the task all three are built for. The mesh is parsedonce; each library builds its own structure over the same triangles and answers the same queries,
with distances cross-checked (0 mismatches). fcpw is built with its Enoki CPU vectorization
(vectorized MBVH) for a like-for-like float/SIMD comparison;
TriangleMeshDistanceis header-only,double-precision and scalar (a reference point, not a same-precision comparison). Representative:
TriMeshSDF~2.6, fcpw (Enoki) ~3.5,TriangleMeshDistance~10.5 µs/query.Each benchmark has a
GNUmakefile, and every result is verified against a brute-force / independentbaseline. Adds a
Benchmark.rstdocs page (in the toctree),Benchmark/README.md, and aREUSE.tomlblock for the benchmark build files/READMEs (main.cpp carry inline SPDX).
Side-effects
FCPW_USE_ENOKI, vectorized MBVH) — a like-for-likefloat/SIMD comparison with
TriMeshSDF.FCPW_SIMD_WIDTHis aGNUmakefilevariable (default 8 =AVX2; 4 = SSE, 16 = AVX-512) to match the build machine's ISA.
TriangleMeshDistanceis double-precision and scalar (no SIMD) — its query runs in double, so itis not a same-precision comparison; it is included as a widely-used reference point, with distances
cross-checked against
TriMeshSDF.Benchmark/and comparison submodules are not part of CI (they pull external libraries withtheir own build systems), matching how
Integrations/is treated.common-3d-test-modelschanges the default mesh path in the CI-tested Examples (MeshSDF,CSGUnion) and Integrations examples; all were updated and re-verified.
Alternative solutions
were chosen for exact version pinning (better for reproducible benchmarks), per Benchmarks: compare EBGeometry against other NN/geometry libraries (picoflann, nanoflann, ...) #109.
Benchmark/vsIntegrations/Benchmark— went with top-level, since benchmarkingagainst external libraries is distinct from integrating EBGeometry into an application code.
Reviewer checklist (to be completed by a human)
@claude review.