Hacker News new | ask | show | jobs
by zylepe 1692 days ago
The main driver of memory usage (~1.0x the input file size) is storing node locations so we can convert the list of node IDs on each way into lat/lon coordinates. By default nodes are stored in a memory-mapped file so it can run with less than 1.5x the input file size, just slower because of all the page faults. Try running with -Xmx=16g to leave 16GB free for memory-mapped file. it will print % complete as it works through the ways in pass2 so you can decide whether it's worth it to keep waiting.

Also you could try switching from the default --nodemap-type=sortedtable to --nodemap-type=sparsearray. Sorted table uses 12 bytes per node and is more compact for extracts, sparse array uses 8 bytes per node but wastes some storage when there are gaps in the ID space.

There is also an osm.pbf extension that embeds node locations in ways, but you need just as much RAM to convert a file to that format: https://docs.osmcode.org/osmium/latest/osmium-add-locations-...

Once you generate the mbtiles file, you can serve from a much smaller machine.

1 comments

Thanks for this detailed answer.