That's because it assumes 96 dpi for every X11 client and scales it up, even for non-fractional scaling (i.e. @2x) if the experimental support is enabled. There is supposedly work being done to address the problem by using two Xwayland servers, one exposing real dpi for dpi aware clients, and one fixed dpi for all the rest.
Unfortunately, it is highlighted by the fact, that all current Linux browsers are X11 and not Wayland clients, so using it would mean you watch the upscaling ugliness on your display every day.
I'm not trying to imply the situation is perfect, but its way more workable than you are letting on.
In the gnome world, you are stuck with GTK3 apps if you want partial scaling that looks alright, but there are options for web browsers.
* Epiphany has supported Wayland since the time of the dinosaurs. And I know its not a perfect browser, but it (1) looks native (2) starts up in a second, (3) has libva support support for hardware accelerated video, (4) has a built in adblocker.
* Most big name distros have some blessed(like fedora) or less blessed (aur/ppa) builds of the latest stable version of Firefox with baked in wayland support.
Epiphany for all its virtues is simply not that fast of a browser when compared to Firefox or Chrome. If I ever find myself with enough free time to dedicate to an OSS project, it will be epiphany. Never has there been such a diamond in the rough.
It's very serviceable if you are on a fast computer, and hardware accelerated video in a linux web browser is such a treat, but I bet any given release of the big boys has more patches in it than a year's worth of epiphany builds.
As for firefox-wayland, I've seen that fat bug tracker, and it doesn't have webrender or many of the fancier performance stuff working, but it does run well. I've been bouncing between the two browsers for a few months now and there doesn't seem to be any showstopping issues. It's just not as shiny as the real deal firefox.
Wayland Firefox: For me, it doesn't work at all with scaling, and on another machine with no scaling, opening new window (in the sense of a new wayland surface, so that includes dialogs) could take 30-60 seconds, during which Firefox would be completely unresponsive (this was fixed in v65 yesterday).
Hw accelerated videos: some distributions (fedora, suse, arch?) have patched Chromium with VA-API support. On some GPUs, the allow_rgb10_configs raises it's ugly head though. In also works only in native X11, not in Xwayland. Other than that, it is only way to watch youtube during the more intensive compiles :)
Unfortunately, it is highlighted by the fact, that all current Linux browsers are X11 and not Wayland clients, so using it would mean you watch the upscaling ugliness on your display every day.