| HN Mirror

Yeah, Wikia in aggregate was in the top 50, maybe a top 20 site at various points. Wikia was built on caching. From my memory, about 99% of page views hit some kind of cache. If that dropped down to 97%, servers started to suffer. It's good to remember that the Fastly CDN company is a spinoff of Wikia, it was developed internally there first. Without that (varnish cache plus lots of memcache) Wikia would not have been able to handle the traffic. Mediawiki is horribly inefficient and one reason why Wikia was attractive as a host was that we had figured out a bunch of tricks to run it efficiently. The default configuration of mediawiki/wikipedia is real bad. Bigger independent wikis just couldn't handle the scale and many of the best independent wikis moved there for that reason. Just as one example, every link/url on a page hits a hook/callback that can call into an extension literally anywhere in the code base, which was several million lines of PHP code. I remember the "Batman" page on the DC wiki used to take several minutes to render a new copy if it fell out of the cache. That was one page I used for performance optimization tests. The muppet wiki and the lyrics wiki also had huge performance issues and fixing them was some of the most fun engineering work I've done. Every useful feature had some kind of horrible performance side effect, so it was always a fun puzzle. I also hate landing on a Fandom wiki now but thanks to the actual editors, it's still got some good content.