Yeah it seems like the real bottleneck is that in these super dense areas, there might be a ton of shards in a mid-size radius. This is inevitable since in the middle of Manhattan, there are just a lot of people in a small area.
One solution would be to scale their recommendation radius with density; no need to look farther than 5 miles for true love in Manhattan, but in Wyoming you may have to drive a bit.
The flip side though is that you don't need to show everyone to everyone, you just need to make sure people have enough other people to swipe. This opens up all sorts of other optimizations, and makes the use of an actual search engine seem a bit weird to me.
One solution would be to scale their recommendation radius with density; no need to look farther than 5 miles for true love in Manhattan, but in Wyoming you may have to drive a bit.