[citricsquid]

As the person ultimately responsible for the Minecraft Wiki ending up in the hands of Fandom, it is great to see what Weird Gloop (and similar) are achieving. At the time of selling out, the Minecraft Wiki and Minecraft Forum cost tens of thousands of dollars per month to run and so it didn't feel too much like selling out, because we needed money to survive[1]. 15 years later, the internet is a different place, and with the availability of Cloudflare, running high-traffic websites is much more cost effective.

If I could do things over again, on today's internet, I like to believe Weird Gloop is the type of organisation we would have built rather than ending up inside Fandom's machine. I guess that's all to say: thank you Weird Gloop for achieving what we couldn't (and sorry to all who have suffered Fandom when reading about Minecraft over the years).

[1] That's a bit of a cop out, we did have options, the decision to sell was mostly driven by me being a dumb kid. In hindsight, we could have achieved independent sustainability, it was just far beyond what my tiny little mind could imagine.

[preciousoo]

You and your team made (a good portion of) my childhood. I remember spending nights studying all the potion recipes and enchantment odds. Thanks for all you did

[Svip]

I was approached about a decade ago to combine The Infosphere with then Wikia's Futurama wiki. I asked it was possible to do a no-ads version of the wiki, and while initially they seemed like that might be possible, they eventually said no, and so we said no. So now there are two Futurama wikis online. I still host The Infosphere, haven't checked the Fandom one in years.

Fortunately for me, Futurama isn't as popular as Minecraft (for some reason!), so I've been able to pay out of my own pocket.

[Svip]

A bit of a follow up to this; after a bit of thought, I am considering reaching out to Weird Gloop. I do not feel I am able to give The Infosphere the care that it deserves. And with Futurama back on Hulu, we are naturally seeing an uptick in activity. We have a very restrictive sign up in place, because I don't have time to moderate it anymore. It keeps the spam down, yes, but also new users away.

Note: The reason I'm writing I'm _considering_ reaching out and not just straight up reaching out is because the domain itself has a different owner than me, and I want to make sure they are also approving of this decision.

[stavros]

What kind of costs are associated with something like this, and what sort of visitors are you getting? I'm wondering what kind of infrastructure you need.

[Svip]

Importantly, I have since set up Cloudflare before the website to help. I am just using their free tier, but looking at their analytics, they say we got about 350k HTTP(S) requests in the last 24 hours.

Had it not been for Cloudflare, I am not sure my server could have handled that. Before I did that, I set up Varnish as a cache provider for users who are not logged in. That is effectively the second line of defence now.

The server itself is a dedicated server at Hetzner. I use the server for a bunch of other things, that see nowhere near the same activity as the Infosphere, and I also use it for my personal screen+irssi setup. But all in all, the server costs me about 50 euros a month.

Though, again, Cloudflare is basically the single most important reason it's not costing me more, and why I have not needed to hand it over.

[stavros]

Ah OK, that's basically exactly the setup I'd use as well. Surprising that the server alone couldn't handle the traffic, as the sibling says, 4 rps isn't that much when you cache (cache hits are basically free).

I imagine 90% of the traffic (or more) is anonymous users, which can be cached, doesn't Varnish handle that without breaking a sweat?

[account42]

4 requests per second is absolutely something even a cheap VPS should be able to handle, even if you double that for peak load. You just need to put caching in front of everything dynamic.

Disappointing for people just carelessly giving Buttflare the keys to the kingdom and effectively excluding alternative Browser users without considering other options.

[cduzz]

An off-hand reference to "350k/day" shouldn't be naively translated to "4 per second"

350k/ day likely means sometimes it's 3.5 million, all smashed into a 30 minute period of time, because some nitwit linked to my site.

And then, I get paged about "my site being down" and I have to stop hanging out with my friends or family and fiddle around with things I don't want to fuzz with. Or maybe it just breaks and doesn't self heal and it is offline for a week until I notice it and fix it, and by then people all think the site's gone.

Anyhow, sure, maybe people not wanting to devote their lives to devops fanfic is something that can "just be solved with this simple trick cloudflare hates" but maybe not.

[Svip]

To keep it simple, this is also a response to your sibling: https://news.ycombinator.com/item?id=41807715

It's been a long time since I switched to Cloudflare. Looking through my email archive, it was December 2015. I uncovered an old discussion[0] about the switch, but it only seems to highlight that the server is slow.

But I think it speaks to my lack of skill in this area. I have no actual professional training in system administration, and entirely autodidactic in this area. Though it sounds like Weird Gloop can also provide guidance in these matters rather than simply taking on the hosting. I won't deny that at times I have felt defeated, and that may truly have been my reasoning for switching to Cloudflare.

Though this post and response so far have given me hope.

[0] https://theinfosphere.org/Table:Server_news! (the exclamation point is part of the URL, in case HN ignores it)

[echelon]

Their growth people emailed me again and again and tried to do the same with StrategyWiki decades ago.

Here's one of their emails:

> [Redacted] mentioned that your site was very cool - and that you're heading off to college. As you may know, Wikia is founded by Jimmy Wales (of wikipedia fame) and we are trying to become THE resource for all gamers

> I was wondering if you'd consider moving over to wikia now that you're going to might have less time with your studies. As an incentive I can point to a few things that might make the move easier

> 1. We have cool new software - gaming.wikia.com lets users do more blog-like contributions in addition to wiki editing - new social networking tools on the wiki - see our test server at http://sports.box8.tpa.wikia-inc.com/index.php?title=User:[R...

> 2. We could also hire you to help moderate the strategy wiki and other wikis if you need some beer and pizza money :-)

> 3. or we could offer to pay all the hosting costs and share some of the ad impressions/revenue with u

> If nothing else, I'd love to chat by phone and get to know you better.

> Let me know if that'd be ok :-)

[wolpoli]

This is basically an offer to buy your business for $0 and we might hire you as a contractor. It's a bad deal. I mean Jimmy Wales himself wouldn't have accepted this for Wikipedia.

[Aeolun]

Ugh, this sits very close to ‘exploiting cheap labor’, and if the ads didn’t, it makes me want absolutely nothing to do with the site.

It’s so upbeat too. I can totally see someone that doesn’t know better being taken in.

[II2II]

Cripes, that sounds creepy and exploitive. I'm pretty sure it would have raised more than a few red flags in my mind, even as a teenager about to head off to college. (Granted, I was a wee bit uptight at that age.)

[karaterobot]

We would like 2 squeeze what u made until all the money runs out + underpay u :-)

[account42]

> As you may know, Wikia is founded by Jimmy Wales (of wikipedia fame)

And Jimmy should be ashamed about being involved with Fandom/Wikia. Then again, he's also not ashamed about begging from third-world people and others much less well off as himself.

[permo-w]

the way this is phrased reminds me so much of IOI from Ready Player One

[babypuncher]

The Infosphere has always been one of the best fan wikis out there, thank you for your hard work (and for not selling out to Fandom)

[ryukoposting]

I remember reading the Minecraft wiki back in the early 2010s, back when Fandom was still Wikia. It would have been much more appealing at the time than it is today - not just for the reasons you list, but because Wikia actually kicked ass in the early 2010s. It was sleek, modern, and easy to use. And today, it isn't.

[epiccoleman]

Every time I wind up on some garbage Fandom page I reminisce about the good old days of Wikia. I remember many a fun night trawling through pages while playing Fallout or Skyrim or whatever - all the information you could ever need, right there at your fingertips. It's an ethos you don't see so much on the modern net.

[iamacyborg]

It’s funny that people are now looking back at wikia fondly because at the time most folks thought it was full of ads and shit. To the point where Curse/Gamepedia managed to get serious market share by not screwing with the community in the same way at the time.

Funny how they somehow managed to make it worse.

[ogurechny]

And before that, Wikipedia itself was Wikia, with lists of cheat-codes for games or paragraphs of inspired “original research” in articles.

Or the complete plot of “Harry Potter”, as seen in this 20 year old artifact:

https://harryfansowned.ytmnd.com/

[Aeolun]

I remember thinking that wikia sucked at the time, but at least it didn’t actively hinder me from finding what I was looking for. I just don’t open fandom pages because it locks up my phone.

[hinkley]

How did Curse end up making money?

[yifanl]

I assume they didn't, which is why they were bought by Twitch.

[iamacyborg]

Lots of ads across their wiki and other community websites and D&D Beyond was remarkably successful.

[mossTechnician]

Wikia is a great example of enshittification - provide great value to users, then take it away from users and hand it to other businesses (eg advertisers), then take it away from businesses too.

Will Weird Gloop inevitably suffer the same fate? I hope not.

[diggan]

> Will Weird Gloop inevitably suffer the same fate? I hope not.

Unless explicitly structured to prevent it, my bet is it will. If it's backed by a for-profit entity, it'll eventually need to turn a profit somehow, and users/visitors are the first to lose their experience at that point.

However, if Weird Gloop is a properly registered non-profit with shared ownership between multiple individuals, I'll be much more likely to bet it won't suffer the same fate.

I skimmed around a bit on the website to try to get an answer to if it is an non-profit, but didn't find anything obvious that says yes/no.

[cookmeplox]

We're already turning a profit! And there are no third-party investors (or debt) – it's all controlled by wiki people[1]

[1] https://meta.weirdgloop.org/w/Weird_Gloop_Limited

[diggan]

Aw, I take that as it is in fact a for-profit company already.

Regardless, I wish you luck for the future! May you not go down the almost inevitable enshittification hole.

[robotnikman]

At least it is a private company though, meaning they are are required to make constant year over year gains for shareholders and investors. They have much more control over where the company goes and how it operates.

[adw]

It’s a company limited by guarantee, which is the structure you use in the UK for non-charity non-profits.

[cinntaile]

If it started that way, I'd say it's less likely to end up "bad". Compared to non-profit websites that get sold to ad businesses.

[Imustaskforhelp]

How is it making money?

[cookmeplox]

We have services agreements with the League of Legends and RuneScape developers, and we run 1 ad (below-the-fold, not in EU/UK) on the RuneScape wikis. This covers all expenses (including 5 staff) by a pretty healthy margin

[hiatus]

It is described in the linked article.

> The company primarily relies on three streams of revenue: user donations, serving ads on select Weird Gloop wikis, and a contract with Jagex that includes a fee to cover hosting and administration costs.

[madeofpalk]

Can we flip it? Some companies are explicitly structured to guarantee enshittification.

Venture capital/private equity is what causes this. We've been poisoned to believe that websites should exist purely to achieve hyperscale and extract as much money as possible. When you look at the real physical world there are tons of small "mom and pop" businesses that are content with being self sustainable without some special corporate structure to legally require that.

Maybe websites could be the same?

[SoftTalker]

There are millions of websites like that. They don't show up on the first page of search results, so nobody finds them.

[godshatter]

Mainly because our biggest search engine is owned by an ad agency.

[Aeolun]

I work for private equity, and while we have a lot of layoffs, we don’t necessarily pursue short term gains (at least, as far as I can determine not as a factor of being PE anyway)

[zellyn]

The article explicitly covers this question. Looks like they're setting up explicit legal(?) agreements. One key point is the domain name: minecraft.wiki, for example, not a subdomain of something owned by Weird Gloop. So the wiki can leave if it wants to.

[mossTechnician]

Does that mean that to the users of these wikis, the switching costs[1] of the backend would basically be zero (one day they might just end up on a different server with the same content), while on the administrators' side the switching costs are at a reasonable minimum?

[1] a variable in whether something can be enshittified, via https://en.wikipedia.org/wiki/Enshittification#History_and_d...

[Nadya]

To my understanding wikis can take all their data, host it themselves, point the domain to their new hosting, and the move would be entirely invisible to end users if done properly and the quality of the hosting infrastructure wasn't considerably worse.

Observant users might notice the removal of any Weird Gloop branding but otherwise the only way people would know if the wiki itself announces the move or performance of the wiki becomes noticeably worse.

And Weird Gloop won't do what Fandom does and keep a zombie copy of your wiki online. So you won't be competing with Weird Gloop wiki traffic to reclaim your traffic. In fact, the obligations they agree to forbid it.

Reading the Minecraft.wiki Memorandum: https://meta.minecraft.wiki/w/Memorandum_of_Understanding_wi...

Upon termination by either party, Weird Gloop is obligated to:

- Cease operating any version of the Minecraft Wiki

- Transfer ownership of the minecraft.wiki domain to the community members

- Provide dumps of Minecraft Wiki databases and image repositories, and any of Weird Gloop's MediaWiki configuration that is specific to Minecraft Wiki

- Assist in transferring to the community members any domain-adjacent assets or accounts that cannot reasonably be acquired without Weird Gloop's cooperation

- This does not include any of Weird Gloop's core MediaWiki code, Cloudflare configuration, or accounts/relationships related to advertising or sponsorships

This sort of agreement means Weird Gloop is incentivized to not become so shit that wiki would want to leave (and take their ad revenue with them) because they've tried to make leaving Weird Gloop as easy as possible.

[mossTechnician]

This is very reassuring. Usually, I assume agreements between different groups will inordinately benefit one party, but this particular agreement sounds like it creates a more level playing field.

And besides, it's not like non-profits are exempt from restructuring and becoming worse. There is no silver bullet.

[cookmeplox]

Yeah - it would be on the same domain, so way users access it wouldn't change at all.

If any of the wikis we host want to leave, we'd provide them with a database dump. The admins would have to configure all of their own MediaWiki stuff of course, but I figure that's a pretty reasonable switching cost.

[treflop]

I find this tends to happen when something passes on from its creator to someone else. Wikia/Fandom has passed hands a bit.

Other people just have very different values and the direction of an organization reflects this.

[stonemetal12]

Thanks (seriously). Fandom may not be great, but you could have said I don't want to foot the bill, turned off the servers and walked away. Then the community would have lost every thing. Leaving it with Fandom gave Weird Gloop something to start with instead starting from scratch.

[beAbU]

I can't imagine that this would have happened, like ever. The wiki was basically essential reading prior to starting to play Minecraft, especially in the early days. I think most the crafting recipes were documented by the developers themselves during those days.

If they killed the wiki, they would have killed their userbase.

[lolinder]

citricsquid wasn't a Mojang employee. This whole thing is and always has been community-run [0], so the "they" in "if they killed the wiki" is not the same as the "they" that was selling Minecraft.

Now, one could reasonably ask why Mojang/Microsoft didn't (and I'm assuming don't) foot the bill for the manual that is an essential part of their game.

[0] https://minecraft.wiki/w/Minecraft_Wiki_(website)

[Washuu]

Hey Criticsquid!~ \(￣︶￣*\)) It's Azxiana[1].

I hate that MCW ultimately ended up with Fandom in the end. Keeping MCW and the other wikis running smoothly was essentially my one huge passion in my life that I lost after Fandom acquired Curse. No one wanted it to happen that way. Even internally at Curse/Gamepedia we were all devastated when we learned that the company was buying bought out by the rival we were striving to overcome all those years. I am so glad to see after the past few years that the wikis are finally healing and going to places that are better for them.

[1] I'm the tech lead/manager that worked on Gamepedia at Curse that administered Minecraft MCW for many years before Fandom bought Curse in December 2018. I'm just writing this here since I figure other readers won't have any idea. ヾ(≧▽≦*)o

[why_at]

One thing I find interesting about playing video games in modern day is that with the proliferation of Wikis, there is assumed to be some kind of third party guide for every game. Especially in smaller/newer games it seems like developers sometimes don't bother putting necessary information in the game at all because they don't have the person-hours for it.

For instance, back when I first played Minecraft in Alpha the only ways to find the crafting recipes was through a wiki, or trial and error.

It's nice that it makes development easier, but I wonder if this trend is making it harder for new people to get into video games, since it's hardly obvious if you're not used to it.

[mhink]

> One thing I find interesting about playing video games in modern day is that with the proliferation of Wikis, there is assumed to be some kind of third party guide for every game. Especially in smaller/newer games it seems like developers sometimes don't bother putting necessary information in the game at all because they don't have the person-hours for it.

While this may have become more of a norm in recent years, online communities with community-supported guides have definitely been around since before wikis were common in the gaming community: most notably at gamefaqs.com. To this day you can still find plaintext walkthroughs for thousands of games, written 25 years ago by pseudonymous authors.

Which isn't exactly to dispute your point, just waxing nostalgic about the good ol' days. The RPG Maker 2000 forum was basically my introduction to programming, waaay back in the day.

[BlueTemplar]

I'm more curious at the state of things before online and gamefaqs.com ?

I do remember downloadable (and maybe also bundled with game magazines ?) game tricks encyclopedias in the Windows help file format.

[egypturnash]

Magazines and guidebooks.

Video game magazines would regularly publish short walkthroughs and maps, as well as tips on common places to be stuck in popular games, and cheat codes.

Guidebooks were found in stores next to the games, they were typically slim, full-color affairs full of screenshots and production art, with complete lists of all the stuff you could do in the game. Full walkthroughs, item statistic charts, locations of the 52 Secret Gears you need to collect to build the Wind-Up Sword to achieve the secret ending, etc, etc. Here's a photo of someone's collection of a bunch of them: https://www.reddit.com/r/originalxbox/comments/12rsvll/seems...

[christianqchung]

I don't really know how exploratory most games are compared to old Minecraft. Some games like Stardew Valley have certain things that are much easier to do because of third party wikis but I don't think the same is true of a lot of games in the same way it was for Minecraft.

[LeifCarrotson]

I picked up Stardew Valley a few months ago for the first time, and consciously chose not to use the wiki. I'm obviously way behind where I would be had I used the wiki, but it's been fun figuring out what works by myself.

One game I recently got which has great exploratory potential is Shapez 2. The in-game help is amazing.

[BlueTemplar]

With emphasis of both free and easily accessible ones... can you still even buy third party game guides in book form ?

[oreally]

> with the availability of Cloudflare, running high-traffic websites is much more cost effective.

sidetrack but how does cloudflare make things cost effective? wouldn't it be cheaper if i just hosted the wiki on a simple vps?

[citricsquid]

More than a decade has passed since then so I am stretching my memory. At peak we were serving in the region of 10 million page views per day which made us one of the most popular websites on the internet (Minecraft was a phenomenon and every Minecraft player needed the wiki). We were probably the highest traffic Wiki after Wikipedia. Nowadays Cloudflare could absorb most traffic because of the highly cacheable nature of it, but at the time, Cloudflare didn't exist, and every request hit our servers.

[owyn]

Yeah, Wikia in aggregate was in the top 50, maybe a top 20 site at various points. Wikia was built on caching. From my memory, about 99% of page views hit some kind of cache. If that dropped down to 97%, servers started to suffer. It's good to remember that the Fastly CDN company is a spinoff of Wikia, it was developed internally there first. Without that (varnish cache plus lots of memcache) Wikia would not have been able to handle the traffic. Mediawiki is horribly inefficient and one reason why Wikia was attractive as a host was that we had figured out a bunch of tricks to run it efficiently. The default configuration of mediawiki/wikipedia is real bad. Bigger independent wikis just couldn't handle the scale and many of the best independent wikis moved there for that reason. Just as one example, every link/url on a page hits a hook/callback that can call into an extension literally anywhere in the code base, which was several million lines of PHP code. I remember the "Batman" page on the DC wiki used to take several minutes to render a new copy if it fell out of the cache. That was one page I used for performance optimization tests. The muppet wiki and the lyrics wiki also had huge performance issues and fixing them was some of the most fun engineering work I've done. Every useful feature had some kind of horrible performance side effect, so it was always a fun puzzle. I also hate landing on a Fandom wiki now but thanks to the actual editors, it's still got some good content.

[immibis]

How much of your server load was Grand Exchange Market Watch?

[BlueTemplar]

I bet, being by some counts the most popular video game ever - but which also makes it kind of a bad example to use when talking about wikis.

By definition, very few wikis will have to deal with becoming one of the most popular websites. (And as you say, at that point one should be able to figure out funding.)

[pjc50]

Cloudflare get the best deals on bandwidth. It will usually be cheaper to serve a terabyte from Cloudflare than to do it yourself: you could probably run the wiki on the free plan!

[account42]

Perhaps, but VPS traffic prices are also already a lot better than "big cloud" traffic prices, especially if you choose your VPS provider with that in mind. And once your traffic is large enough there are also options where you pay for a fixed pipe instead of a transfer amount.

[diggan]

> Cloudflare get the best deals on bandwidth.

If you want to pay for bandwidth then yeah, CloudFlare is a great option.

Otherwise, if you like the experience of not paying per GB/TB, go for a dedicated server with unmetered connection that has the same price every month, regardless.

[rjmunro]

Cloudflare don't charge per GB/TB. You get unlimited bandwidth even on their free plan. The problem with paying per GB is that it's in the CDN's interest for you to get a DDOS attack so they can charge you for all the bandwidth. It's in Cloudflare's interest to reduce DDOS attacks and unwanted bot traffic because it costs them bandwidth, not you.

[wpietri]

Your point on interest is spot on.

I moved a few of my personal websites to AWS's CloudFront and it cost me like a buck a month, way cheaper than maintaining a virtual server to do it. Except that somebody somewhere decided to try their DDOS tool on one of them for a few hours in the middle of the night, and I got a bill for $2541.69.

Eventually they credited it, but it was not a fun ride, and decided that I was done using a CDN with misaligned incentives: https://sfba.social/@williampietri/111687143220465824

[Aachen]

> it's in the CDN's interest for you to get a DDOS

What kind of conspiracy is this? As if anyone charging for bandwidth hopes to get their infrastructure attacked

[ezrast]

The whole point of systemic incentives is that there is no conspiracy. Nobody wants a DDOS and every large provider will have people genuinely working to avoid them. But every time there is an opportunity to allocate resources, the team that gets to frame their return on investment in terms of real dollars will always have an edge over one whose value is realized only in murky customer satisfaction projections. Over the lifetime of a company, the impact of these decisions will add up with no need for any of the individuals involved to even be aware of the dynamic, much less conspire to perpetuate it.

[0cf8612b2e1e]

Why not? They have the capacity they could absorb nearly any kind of attack without blinking.

[KomoD]

You don't need to pay anything to run TBs through Cloudflare, you could use the free plan.

Rent VPS or managed hosting or host wherever you want, proxy it with Cloudflare on the free plan, Cloudflare caches it.

[robertlagrant]

It's more like: if you have a website that (sometimes) gets a lot of traffic, do you want Cloudflare to cache it and serve it with very few hits to your cheap server, or do you want your compute costs to expand to cope with the requests?

[diggan]

> do you want Cloudflare to cache it and serve it with very few hits to your cheap server, or do you want your compute costs to expand to cope with the requests?

Usually you have something like a platform/tool/service that is mostly static requests that could be cached, with some dynamic requests that couldn't, as they're CRUD requests or similar.

If your struggling to serve static content, then do go ahead and slap Cloudflare on top of that bad boy and probably your visitors will be a bit happier, instead of upgrading from a cheap VPS.

If you're struggling to serve the dynamic requests, Cloudflare/CDN won't matter because these things actually need to be processed by your backend.

So instead of trying to shave 50ms off from my simple static requests with a CDN, I'd much happier to optimize for all the requests, including the "dynamic requests" that need to hit the backend anyway.

I'll still go for a dedicated server with proper connection and performance rather than a shitty cheap VPS with a CDN in front off it.

[pjc50]

> If you want to pay for bandwidth then yeah, CloudFlare is a great option.

The free plan is a lot bigger than you think.

> dedicated server with unmetered connection

And where have you found one of those with reasonable pricing?

[diggan]

Hetzner is pretty cheap, but only offers Europe location for their dedicated servers last time I checked. For more locations, DataPacket is nice, although a bit more expensive.

[Ambroos]

If you can run your application on Cloudflare Pages / Workers with Cloudflare's storage/DB things, it really gets dirt cheap (if not free) and very fast. And even without that, Cloudflare's caching CDN is very good, very cheap and very easy.

[bombcar]

Ten years ago bandwidth was expensive. Still is, even if not as much. A simple VPS gets overwhelmed, but a simple VPS behind cloudflare can do quite well.

[thinkmassive]

    s/cloudflare/a CDN/

[pornel]

Cloudflare caches pages at many many datacenters, often colocated with large ISPs.

This lets Cloudflare deliver pages from their local cache over local links (which is fast and cheap), instead of fetching the data every time across the world from wherever the VPS is located.

[jchw]

In all fairness, running modest to large MediaWiki instances isn't easy. There's a lot of things that are not immediately obvious:

- For anything complex/large enough you have to set `$wgMiserMode` otherwise operations will just get way too long and start timing out.

- You have to set `$wgJobRunRate` to 0 or a bunch of requests will just start stalling when they get assigned to calculate an expensive task that takes a lot of memory. Then you need to set up a separate job runner in the background, which can consume a decent amount of memory itself. There is nowadays a Redis-based job queue, but there doesn't seem to be a whole lot of documentation.

- Speaking of Redis, it seems like setting up Redis/Memcached is a pretty good idea too, for caching purposes; this especially helps for really complicated pages.

Even to this day running a Wiki with an ambient RPS is kind of hard. I actually like MediaWiki because it's very practical and extensible, but on the other hand I know in my heart that it is a messy piece of software that certainly could make better use of the machine it's running on.

The cost of running a wiki has gone down over time in my experience though, especially if you are running things as slim as possible. A modest Digital Ocean machine can handle a fair bit of traffic, and if you wanted to scale up you'd get quite a boost by going to one of the lower end dedicated boxes like one of the OVHcloud Rise SKUs.

If anyone is trying to do this I have a Digital Ocean pro-tip. Don't use the Premium Intel boxes. The Premium AMD boxes are significantly faster for the money.

One trap I also fell into was I thought it might be a good idea to throw this on a hyperscaler, you know, Google Cloud or something. While it does simplify operations, that'll definitely get you right into the "thousands of dollars per month" territory without even having that much traffic...

At one point in history I actually felt like Wikia/Fandom was a good offering, because they could handle all of this for you. It didn't start out as a bad deal...

[noen]

This is so true.

I adopted mediawiki to run a knowledge base for my organization at Microsoft ( https://microsoft.github.io/code-with-engineering-playbook/I... ).

As I was exploring self-host options that would scale to our org size, it turned out there was already an internal team running a company wide multi-tenant mediawiki PLATFORM.

So I hit them up and a week later we had a custom instance and were off to the races.

Almost all the work that team did was making mediawiki hyper efficient with caching and cache gen, along with a lot of plumbing to have shared infra (AD auth, semitrusted code repos, etc) thst still allowed all of us “customers” to implement whatever whacky extensions and templates we needed.

I still hope that one day Microsoft will acknowledge that they use Mediawiki internally (and to great effect) and open-source the whole stack, or at least offer it as a hosted platform.

I tried setting up a production instance af my next employer - and we ended up using confluence , it was like going back to the dark ages. But I couldn’t make any reasonable financial argument against it - it would have taken a a huge lift to get a vanilla MW instance integrated into the enterprise IT environment.

[bawolff]

Microsoft did open source a bunch of their mediawiki extensions. https://github.com/microsoft/mediawiki-extensions

Last i heard though they were moving off it.

[noen]

Nice!! Made my day. Not sure how they can move off of it, partly because there’s no alternative that has a fraction of the capability

[bawolff]

The rumour i heard is they were making their own custom thing.

There was some rumours that they were unhappy about mediawiki's response to patches they submitted (they made a bunch around accessibility). However i looked through their patches at one point when this rumour started flying around and it looked like most were merged. Those that weren't generally had code review comments with questions or pointing out mistakes which were never replied to. I sort of suspect the patch thing was some sort of internal excuse because the team involved wanted to make their own thing.

Regardless, im really happy they decided to open source their extensions and it was nice to see that they put in effort to upstream core patches.

[noen]

Yeah the team running the platform internally was amazing to work with and did incredible work with just a handful of resources.

The efficiency for scale of mediawiki is hard to beat.

[account42]

A lot of things should be solved by having (micro)caching in front of your wiki. Almost all non-logged in requests shouldn't even be hitting PHP at all.

[jchw]

In my experience this hasn't been necessary yet on anything I've ran. I know WMF wikis run Varnish or something, but personally I'm trying to keep costs and complexity minimal. To that end, more caching isn't always desirable, because RAM is especially premium on low-end boxen. When tuned well, read-only requests on MediaWiki are not a huge problem. The real issue is actually just keeping the FPM worker pool from getting starved, but when it is starved, it's not because of read-only requests, but usually because of database contention preventing requests from finishing. (And to that end, enabling application-level caching usually will help a lot here, since it can save having to hit the DB at all.) PHP itself is plenty fast enough to serve a decent number of requests per second on a low end box. I won't put a number on it since it is obviously significantly workload-dependent but it would suffice to say that my concerns with optimizing PHP software usually tilt towards memory usage and database performance rather than the actual speed of PHP. (Which, in my experience, has also improved quite a lot just by virtue of PHP itself improving. I think the JIT work has great potential to push it further, too.)

The calculus on this probably changes dramatically as the RPS scales up, though. Not doing work will always be better than doing work in the long run. It's just that it's a memory/time trade-off and I wouldn't take it for granted that it always gives you the most cost-effective end result.

[bawolff]

Varnish caching really only helps if the majority of your traffic is logged out requests. Its the sort of thing that is really useful at a high scale but matters much less at a low scale.

Application level caching (memcached/redis/apcu) is super important even at a small scale.

Most of the time (unless complex extensions are involved or your wiki pages are very simple) mediawiki should be io-bound on converting wikitext -> html (which is why caching that process is important). Normally if db is healthy, db requests shouldn't be the bottle neck (unless you have extensions like smw or cargo installed)

[jchw]

Most of MediaWiki seems to avoid too much trouble with contention in the database, but I was seeing it prior to enabling application-level caching. It seemed to be a combination of factors primarily driven by expensive tasks in the background. Particularly complex pages can cause some of those background tasks to become rather explosive.

[tempest_]

Have any of Intels server offerings been "premium" since epyc hit the scene?

I just assumed they were still there based on momentum.

[jchw]

With Digital Ocean the cpuinfo is obfuscated so figuring out exactly what you're running on requires a bit more trickery. With that said I honestly assume that the comparison is somewhat older AMD against even older Intel, so it's probably not a great representation of how the battlefield has evolved.

That said, Digital Ocean is doing their customers a disservice by making the Premium Intel and Premium AMD SKUs look similar. They are not similar. The performance gap is absolutely massive.

[Nux]

> At the time of selling out, the Minecraft Wiki and Minecraft Forum cost tens of thousands of dollars per month to run.

What kind of decisions got you in that position? Hard to phatom.

[hinkley]

One of the things on my todo list is to spend some solid time thinking about load-shedding, and in particular tools and methods for small or hobbyist projects to practice it. Like what do you turn off on the site when it's the 15th of the month and you're already at 80% of your SaaS budget?

Like maybe if a request for an image doesn't result in a 304, instead of sending a 200 response you redirect to lower res versions, or just 429 out. How much throttling do you do? And do you let bots still run full speed for SEO reasons or do you do something else there?

[Arch-TK]

You say you were a kid when you sold it. I could have sworn you weren't from conversations we had on IRC at the time.

Although I most assuredly was a kid.

[fwip]

"Kid" doesn't really have a hard cutoff. When you're 15, 12-year-olds are kids. When you're 30, 20-year-olds are kids.

[citricsquid]

I was a teenager at the time. I'm in my mid 30s now, it feels like I was a kid back then.

[RGamma]

To be fair a lot of wikis' and internet cultural places' continuity woes would be mitigated by making it easier to decentralize hosting or at least do a git pull. Wikis especially don't tend to be that large and their S/N is quite high, making them attractive to mirror.

[jagermo]

holy crap that minecraft wiki is fast now. I actually stopped going to fandom because it was so slow.

[Dwedit]

Ah Cloudflare, where you constantly get captchas for attempting to read a web page.

[matt_heimer]

That's up to the site owner.

For example I configured my osdev wiki (mediawiki based) so that the history and other special pages get the Cloudflare test but just viewing a page doesn't trigger it. OpenAI and other bots were generating way too much traffic to pages they don't need.

Blame the bots that are DDOS'ing sites for the captchas.

[BlueTemplar]

Wait a minute, DDoSing is illegal, how come OpenAI et al. haven't gotten sued to the ground yet ??

[ruune]

Not a lawyer, I'm guessing here. I'd assume the intention matters a lot. Scrape bots don't intend to cause trouble, they intend to get your data (for free). Same way as when some famous person tells people on Twitter to visit a website or when some poor blog gets the hug of death from HN. The intention wasn't to bring down the site.

Aside from that: is DDosing actually illegal (under US law)?

[BlueTemplar]

Right. Pretty sure it's illegal under EU law(s), and people were already condemned for it (but yes, in case ill intent was proven) - why wouldn't it be illegal under US law - it's basically akin to vandalism ?

(In other news, the Internet Archive got DDoSed today :(

[whstl]

At least they moved away from Google Captchas, which really hates disabling of 3rd party cookies and other privacy-protection measures.

I haven't had a problem with Cloudflare and their new Captcha system since their changed, but I still suffer whenever I see another website using Google Captcha :(

[ChocolateGod]

Ironically its now easier for robots to solve Google Captchas than it is for humans, as evident by the browser extensions that solve them that exists.

[Dwedit]

I used to have a lot of bot spam, but then I mostly foiled them with the world's silliest captcha. Looks like a math problem, but the solution isn't what's required to proceed.

[palunon]

AFAIK most of those just pay a human in a low income country.

[kbolino]

Even better, you can get a captcha before you're allowed to see 404 Not Found.

[yard2010]

The other side of the coin is lizards trying to literally end the internet era with their irresponsible behavior, and hell, making a nice living in the process

[treefarmer]

And god forbid you use a VPN and try to do anything on a Cloudflare site

[theamk]

Cloudflare dropped captchas back in 2022 [0], now it's just a checkbox that you check and it lets you it (or does not).

And this mean that my ancient android tablets can no longer visit many cloudflare-enabled sites.. I have a very mixed feelings about this:

I hate that my tablets are no longer usable so I want less Cloudflare;

but also when I visit websites (on modern computers) which provide traditional captchas where you click on picture of hydrants, I hate this even more and think: move to Cloudflare already, so I can stop doing this nonsense!

[0] https://news.ycombinator.com/item?id=33007370

[fwip]

The checkboxes are also captchas.

[eviks]

but there are more user-friendly captchas than the hydrants, which on average could be better that a total block on the tablets?

[theamk]

total block on _old_ tablets - Android 4.4 specifically, and I am sure many people on HN would be horrified to see those anywhere close to internet. New tablets are fine.

As for "more user-friendly captchas" - I have seen some of those (like AliExpress' slider) but I doubt they will work as well as hydrants. And with new AI startups (1) slurping all the data on the web and (2) writing realistic-looking spam messages, I am sure anti-bot measures would be more important than ever.