Safari does this too.. Apple is on some horse about requiring user intervention for every single location request even from sites you’ve already allowed in the past. It’s annoying.
I think the only button on the dialog is like "rememberer for one day" or something, right? Once you've clicked that, go to preferences -> websites -> locations, change the affected site to "deny" and then "allow", and it's permanent.
macOS does not forget that Safari has location permissions. In fact, they're enabled by default. Chrome however has its location permissions reset so often that they had to add a feature to tell users it was off again.