If the constants change over very long time spans, we could observe this by looking at distant galaxies from billions of years ago. We don’t have a way to make similar observations within our own galaxy.
What if the constants only changed over incredibly small scales, vibrating back and forth between two very similar numbers like a standing wave with extremely small amplitude and wavelength, such that any measurement done on even small scales has trouble seeing anything but the average?
Let's start with the universe since the ignition of the first stars. Your question is also super-interesting in the context of the very early universe, so I'll come back to that further below.
Depending on the constants, with significant fluctuation of them you'd expect spectral line broadening rather than the sharp lines we see in precision interferometry, violations of local Lorentz invariance, different structures in "stacked" spectra (like the Lyman-alpha forest), and instabilities in Keplerian orbits. Present measurement precision of subatomic transition spectra has really boxed you in on this: many physical constants have relative standard uncertainties on the order of 10^-10 or better.
> any measurement ... [sees only] the average
So you'd start wondering: in the limit of infinitesimal fluctuations, is a fluctuating constant just constant rather than an "effective constant"?
Where's there's still wiggle room is in the exact masses of heaver generation standard model particles (top quark, tau mass, W-to-Z mass ratio for example) and somewhat frustratingly Newton's gravitational constant, all of which have relative standard uncertainties worse than 10^-5.
However, assuming cosmic inflation, one might expect incredibly small scale fluctuations in physical constants to be stretched, just like incredibly small scale fluctuations in the densities of matter and radiation. This could lead to later-universe regions of arbitrary size with a significantly different value for one or more physical constants, just like we see regions relatively stuffed with galaxies (filaments) and regions that are relatively empty (supervoids). We'd expect that when we look at different parts of the sky we'd see differences in things like the Lyman-alpha forest, the population and/or spectra and/or light curves of quasars/supernovae/variables, and so on.
So, in order to have the apparently constant physical constants we observe, while keeping your idea that there are tiny fluctuations in them, you'd have to suppress high frequency fluctuations in the constants in the very early universe, because otherwise you'd have to suppress gross effects like different gas and dust chemistry when comparing one galaxy cluster to another.
(The cosmic inflation epoch predates the "freezing-out" of some of the physical constants, so my thinking is that during inflation there must be some precursor constant(s) that determine(s) the mass of the electron (for example) once there are electrons after the electroweak epoch. Even after inflation the ordinary expansion of the universe can stretch fluctuations enough that (assuming your idea) there is likely to be a directional dependence on precision extragalactic astronomy.)