Skip to content

Core dump with v3.2 #930

Description

@cquest

Converting last planet pbf to mbtiles leads to a core dump...

cquest@t7910:/nvme/pmtiles$ time tilemaker --store data --bbox -180,-90,180,90 --threads 32 optimized.osm.pbf --output planet.mbtiles
Layer place (z0-14)
Layer boundary (z0-14)
Layer poi (z12-14)
Layer poi_detail (z14-14) -> poi
Layer housenumber (z14-14)
Layer waterway (z8-14)
Layer waterway_detail (z12-14) -> waterway
Layer transportation (z4-14)
Layer transportation_name (z8-14)
Layer building (z13-14)
Layer water (z6-14)
Layer ocean (z0-14) -> water
Layer water_name (z14-14)
Layer water_name_detail (z14-14) -> water_name
Layer aeroway (z11-14)
Layer aerodrome_label (z10-14)
Layer park (z11-14)
Layer landuse (z4-14)
Layer urban_areas (z4-8) -> landuse
Layer landcover (z0-14)
Layer ice_shelf (z0-9) -> landcover
Layer glacier (z2-9) -> landcover
Layer mountain_peak (z11-14)
Bounding box -180, -85.06, 180, 85.06
Using osm store file: data
Filename: data/mmap_0.dat, size: 1024000000
Reading shapefile ocean
Reading shapefile urban_areas
Reading shapefile ice_shelf
Reading shapefile glacier
Generated points: 0, lines: 0, polygons: 65812
Reading .pbf optimized.osm.pbf
(Scanning for ways used in relations: 99%)           (12793 ms)
Store size 203G | Block 1351853/1351854 (216468 ms)
SortedNodeStore: 211686 groups, 20503458 chunks, 1914285234 nodes, 9619442964 bytes (1.8% wasted)
SortedNodeStore: 212905 groups, 23492722 chunks, 1946818282 nodes, 10630402580 bytes (2.1% wasted)
SortedNodeStore: 213090 groups, 23491986 chunks, 2002135192 nodes, 10786762692 bytes (2.1% wasted)
SortedNodeStore: 211637 groups, 22966386 chunks, 1933717135 nodes, 10127087876 bytes (1.6% wasted)
SortedNodeStore: 211154 groups, 20852587 chunks, 1332330381 nodes, 7431107094 bytes (1.7% wasted)
SortedNodeStore: 210400 groups, 20194447 chunks, 1685530862 nodes, 8445799998 bytes (1.6% wasted)
Store size 236G | 1/6 Block 151596/151597 (271763 ms)
Store size 269G | 2/6 Block 151596/151597 (300152 ms)
Store size 302G | 3/6 Block 151596/151597 (315682 ms)
Store size 335G | 4/6 Block 151596/151597 (304753 ms)
Store size 367G | 5/6 Block 151596/151597 (291625 ms)
Store size 400G | 6/6 Block 151596/151597 (307322 ms)
SortedWayStore: 23597 groups, 3917382 chunks, 156487697 ways, 1766943322 nodes, 4865553322 bytes
SortedWayStore: 23648 groups, 4244491 chunks, 142205439 ways, 1794777436 nodes, 5313147946 bytes
SortedWayStore: 23667 groups, 4260431 chunks, 150494477 ways, 1820727344 nodes, 5193654054 bytes
SortedWayStore: 23637 groups, 4218291 chunks, 162750745 ways, 1763900513 nodes, 5173780206 bytes
SortedWayStore: 23642 groups, 4028046 chunks, 107311081 ways, 1065980850 nodes, 3665214762 bytes
SortedWayStore: 23590 groups, 3969360 chunks, 141708994 ways, 1480284403 nodes, 4211211364 bytes
Store size 433G | 1/6 Block 1833/1834 (80031 ms)
Store size 466G | 2/6 Block 1833/1834 (134639 ms)
Store size 498G | 3/6 Block 1833/1834 (95181 ms)
Store size 531G | 4/6 Block 1833/1834 (71491 ms)
Store size 564G | 5/6 Block 1833/1834 (15012 ms)
Store size 597G | 6/6 Block 1833/1834 (101119 ms)
Generated points: 120487059, lines: 52990, polygons: 296877946
Attributes: 45916426 sets from 1482094592 objects (229487616 uncached), 1117919232 pairs (180295680 uncached)
Creating mbtiles at planet.mbtiles
indexed 42706553 contended objects
osm: finalizing z6 tile 4096/4096 (317165 ms)
osm: finalizing z6 tile 4096/4096 (0 ms)
indexed 545 contended objects
shp: finalizing z6 tile 4096/4096 (460 ms)
shp: finalizing z6 tile 4096/4096 (0 ms)
collecting tiles: 42619ms, filtering tiles: z0 (1, 0ms) z1 (4, 0ms) z2 (16, 0ms) z3 (64, 0ms) z4 (256, 0ms) z5 (1024, 0ms) z6 (4096, 0ms) z7 (16384, 0ms) z8 (65536, 1ms) z9 (261976, 5ms) z10 (1044530, 24ms) z11 (4149740, 88ms) z12 (16454026, 344ms) z13 (65196276, 1374ms) z14 (258321153, 5642ms)
terminate called recursively
terminate called recursively
terminate called recursively
Abandon (core dumped)

real    51m5,931s
user    1117m15,679s
sys     57m1,326s

Previous runs gave one more detail:

terminate called recursively
terminate called after throwing an instance of 'std::bad_alloc'

I had no problem with past version of tilemaker on the same machine (192GB of RAM).
Could the new version need more memory ?

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions