It shouldn’t be required if the range requests are right and you have a compatible renderer in the client.
To expand a bit -- There are adapters for mapbox(gl/libre) and leaflet, and so on that will allow you to use a .pmtiles directly from the client with a range request supporting server, like s3 or nginx.
I've used protomaps basemaps to generate an Ireland and EU basemaps for a project from OSM data. I still haven't quite figured out the tuning for which features should be shown, combined, or excluded yet, and the generated layer names are sort of, but not quite compatible with similar base layers + styles from mapbox (which uses the same base data, but a different, proprietary conversion). This part takes a reasonable skill as a GIS person and some taste in the design of the feature symbology.
The downside of doing this on S3 directly is that you might wind up publishing a link to a 100 gig file that would cost $10 if someone just downloaded it. Lambda invocations seem cheap compared to that risk. OTOH, throwing it on a hetzner box is quite reasonable.
The transformation of OpenStreetMap features into the basemap is open source and available under a permissive license now at http://github.com/protomaps/basemaps
This is the focus of most development for the next few months - like you said, it's also a matter of aesthetics and context, like if the map is underlying another PMTiles data layer it should be lower contrast then if it's overlaid with pin markers. The end goal is to have a flexible basemap solution adaptable to most apps but you are free to roll your own proprietary style on top of the open source tiles if you'd like (there are already companies doing this)
Yes, that's what I was using for my basemaps, it's quite fast for Ireland on a relatively fast machine (minutes), but Europe as a whole was hours. (hetzner quad core 8 thread ~3yr old intel machine with 64G ram)
I've got a couple of goals, or at least evaluations for goals --
1) Have a stripped down basemap like Carto Light, but with different emphasis for the "bicycling friendly" roads, and selectively higher contrast/emphasis for those minor through roads that discourage car traffic but are perfect for a bike. (e.g. Irish boreens, those 1 lane or double track "paved" L roads that are everywhere outside of the major cities.) I find that I simply can't see the minor roads on apple/google maps when I'm out unless I'm zoomed in to the point that I can barely see a network. (fading eyesight and super low contrast).
2) Another lighter base map, but with transit focused features to compliment some of the open data in Ireland around transit -- the routes and timetables and so on. I'm aiming for a site that can be 100% statically hosted but will show the routes in a more friendly manner than Bus Eireann's site.
3) If it goes well, I might be doing this in a more commercial context for some clients who are currently using mapbox for their basemaps but due to political concerns need to insert in different names/boundaries for disputed areas/features. We'd love to be able to self host, but quality is a major concern there.
The bugs that I'm seeing in the 1 and 2 cases are things like handling the look of freeway interchanges/flyovers at close zooms, consistently getting rivers to show at appropriate zooms because they're into two separate feature types depending on the width, with the narrow bits getting dropped. And the general mismatch between the styling I'm used to from the mapbox converted layer names/feature types and what's coming out of pmtiles/basemaps. The feature bits look like tippecanoe coalesce/drop densest features tuning, but I'm also looking at just dropping out entire feature sets to drop the size of the tiles. That should help the coalesce/dropping behavior as well.
Thanks - for the issues related to freeway interchanges and rivers, it would be most helpful to report these on GitHub using screenshots and links to lat/lon positions, or even links to OSM nodes and ways.
OSM is a freeform dataset and not a cartographic product, so most of the basemap work now and in the future is on getting good results of this transformation for 200+ countries. The mismatch vs. existing maps must exist because we need to ensure to downstream commercial users that all end products of the map generation are openly licensed - that's why this is being pursued as an independent, self-funded project with support from GitHub Sponsors: http://github.com/sponsors/protomaps
Ok, I’ll dig a bit more to get some good test cases with the local data that I’m familiar with, and make sure that it’s not something that I’m doing wrong.
Library author here - yes, one execution happens per tile, if you have CloudFront in front those will only happen when the PoP has a miss.
With 512MB RAM these generally complete in under 100ms so the unit costs of Lambda are within a few multiples of Cloudflare Workers (.5USD / million invocations)
Hey, want to say I'm super excited by protomaps. I'm struggling a bit to convert existing OSM pbf files into the pmtiles format. Specifically, I'm trying to convert this guy https://daylightmap.org/
Do you have any tips for doing so without renting memory in the cloud?
The daylight PBFs are difficult to combine without a lot of RAM, there's no way around it right now. Either do that or hold out for potential newer releases of the data delivered as Parquet files.
Like the other comment says I don't think it's quite that bad but also CloudFront (or whatever CDN you use) is going to cache the response. In my case I'm rendering restaurant locations on a map. I only have X number of items that each have a map of the restaurants they are available at so unless people pan/zoom around a lot I only expect a handful of lamba calls and the resulting tile data will be cached in CloudFront for everyone else.
To expand a bit -- There are adapters for mapbox(gl/libre) and leaflet, and so on that will allow you to use a .pmtiles directly from the client with a range request supporting server, like s3 or nginx.
I've used protomaps basemaps to generate an Ireland and EU basemaps for a project from OSM data. I still haven't quite figured out the tuning for which features should be shown, combined, or excluded yet, and the generated layer names are sort of, but not quite compatible with similar base layers + styles from mapbox (which uses the same base data, but a different, proprietary conversion). This part takes a reasonable skill as a GIS person and some taste in the design of the feature symbology.
The downside of doing this on S3 directly is that you might wind up publishing a link to a 100 gig file that would cost $10 if someone just downloaded it. Lambda invocations seem cheap compared to that risk. OTOH, throwing it on a hetzner box is quite reasonable.