Skip to content

perf: parallelise z6 finalizing and tile collection - #972

Open
dktue wants to merge 2 commits into
systemed:masterfrom
dktue:perf/serial-steps
Open

dktue wants to merge 2 commits into
systemed:masterfrom
dktue:perf/serial-steps

Conversation

@dktue

@dktue dktue commented Oct 31, 2026

Copy link
Copy Markdown
Contributor

Gain (planet, PMTiles, 32 threads): the two serial steps between reading and tile writing drop from 70.5 s to 24.2 s (−66%; −46 s, about 2.7% of the run): collecting tiles 25.4 → 1.1 s (−96%), z6 finalizing 26.3 → 7.3 s (−72%). Total wall 1703 → 1686 s (−1.0%, one run each; the untouched tile phase varies by ±25 s between runs). Output unchanged.

Between reading the .pbf and writing tiles, two steps run on few threads. On the planet (32 threads) they take about 50 s, with most cores idle:

  • finalizing z6 tiles (~23 s at 5–9 busy threads): finalizeObjects sorts the objects of the 4096 z6 tiles one after another. Each sort uses all threads, which suits small extracts (one z6 tile may hold most objects) but leaves the cores idle for the many small tiles of a large extract. A comment in sortOutputObjects already names this as a future improvement.
  • collecting tiles (~28 s on 1 thread): populateTilesAtZoom walks every output object to mark the tiles that hold objects, then marks every z14 tile in the bounding box of each large object (e.g. ocean polygons) together with its 14 parents, one tile at a time.

This PR changes three things.

  • finalizeObjects: z6 tiles holding more than 1/threads of all objects are still sorted one at a time with all threads. The others are sorted in parallel, one thread each, largest first. Each sort copies its z6 tile into a buffer, so the number of objects being sorted at once is capped at 1/8 of all objects (a single tile may always run). The counting sort is stable whatever its chunk count, so the order of objects, and the output, are unchanged.
  • populateTilesAtZoom takes threadNum. Workers take whole columns of z6 tiles, mark tiles in bitmaps of their own (allocated lazily, so each touches only its part), and the bitmaps are merged at the end with the new PreciseTileCoordinatesSet::merge.
  • Large objects: the parents of the tiles in a box are the tiles in the box scaled down, so addCoveredTilesToOutput now calls the new TileCoordinatesSet::setBox once per zoom. PreciseTileCoordinatesSet::setBox fills each row word by word. It ignores the part of a box outside the grid; set() wrapped a y beyond the last row into the next row.

New unit tests check setBox against set() for boxes that start and end mid-word, span several words or touch the edge, and merge against setting both sets' tiles.

Europe, PMTiles, 32 threads, --store --fast

before after
collecting tiles 3.4 s 0.6 s (−82%)
finalizing z6 tiles 7.3 s 4.8 s (−34%)
wall 425.9 s 417.4 s (−2.0%)

Planet, PMTiles, 32 threads, --store --fast

before after
collecting tiles 25.4 s 1.1 s (−96%)
finalizing z6 tiles 26.3 s 7.3 s (−72%)
end of reading → first tile written 70.5 s 24.2 s (−66%, −46 s)
wall 1703 s 1686 s (−17 s, −1.0%)
anon memory peak 13.76 GB 13.73 GB

The two steps save 46 s. The total gains less because the tile phase of this run took 24 s longer (1046 s vs 1022 s). This PR doesn't touch the tile phase, and identical planet runs have varied by tens of seconds before, so I take that as run-to-run noise (one run each).

Output

Germany, compared with compare_area (per-tile area/length per layer and tag group): all layers identical. Only poi differs, which already differs between identical runs.

Baseline for the measurements: master + #961 + #966 + #968 + the precheck and malloc follow-ups.

🤖 Generated with Claude Code

- finalizeObjects: sort small z6 tiles in parallel (one thread each,
  largest first, at most 1/8 of all objects in flight); z6 tiles with
  more than 1/threads of the objects keep the all-threads sort
- populateTilesAtZoom(threadNum): workers mark tiles per z6 column in
  bitmaps of their own, merged at the end (PreciseTileCoordinatesSet::merge)
- addCoveredTilesToOutput: one setBox per zoom instead of setting each
  z14 tile and its parents; setBox fills rows word by word

Planet: collecting tiles 25.4 s -> 1.1 s (-96%), z6 finalizing 26.3 s ->
7.3 s (-72%), wall -1.0%. Europe: collecting tiles 3.4 s -> 0.6 s (-82%),
z6 finalizing 7.3 s -> 4.8 s (-34%), wall -2.0%.
Output unchanged (Germany compare_area).
tile_data.h now includes boost/asio, which pulls in windows.h on Windows,
and windows.h defines `near` as an empty macro: MSVC saw
`const Cell& , ...` in edgeDistBound. Renamed to `nearCell`; nothing else
changes, and the output is the same.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
dktue pushed a commit to dktue/tilemaker that referenced this pull request Nov 15, 2026
- polylabel.h: rename edgeDistBound's parameter `near` to `nearCell`.
  tile_data.h includes boost/asio since systemed#972, which pulls in windows.h on
  Windows, and windows.h defines `near` as an empty macro.
- test/shp_mem_tiles.test.cpp (systemed#979): include shp_mem_tiles.h before
  minunit.h. minunit.h sets _POSIX_C_SOURCE, which on macOS hides ip_mreq,
  and boost/asio (through tile_data.h) needs it.

Renames and include order only; output unchanged.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@dktue

dktue commented Nov 15, 2026

Copy link
Copy Markdown
Contributor Author

The Windows build failed because of this PR: tile_data.h now includes boost/asio, which pulls in windows.h, and windows.h defines near as an empty macro. polylabel.h has a parameter called near (edgeDistBound), so MSVC saw const Cell& , ... and gave up.

34a8100 renames that parameter to nearCell. Nothing else changes, and the output is the same.

One more thing for whoever merges this together with #979: #979's new test/shp_mem_tiles.test.cpp includes minunit.h before shp_mem_tiles.h. minunit.h sets _POSIX_C_SOURCE, which on macOS hides ip_mreq, and boost/asio (through tile_data.h) needs it. Including shp_mem_tiles.h first fixes it. #1001 carries that change.

@dktue
dktue force-pushed the perf/serial-steps branch from 34a8100 to 6f3f49d Compare November 16, 2026 10:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant