Almost none of the IO of that page is cached actually[0]. The only thing I'm sure that is cached is the templates themselves. I'm pretty sure that neither git lookups, nor DB accesses are cached, where you can save time. And mind you, this is served from a single data center in USA, and the latency can already eat up a lot of that. I in Europe have 300ms ping to it, so it might be that you are simply far away from the physical location.
[0] https://git.sr.ht/~sircmpwn/git.sr.ht/tree/master/gitsrht/bl...