[dbrueck]

The anti-cheat problem is long-running and complicated. If you choose not to run anti-cheat because you understand that these are opaque rootkits, good for you! That's a totally, 100% valid choice. But please keep in mind:

  - you are a tiny minority and not the target customer
  - online multiplayer games are an absurdly big business (i.e. there are huge incentives here)
  - no, you can't completely solve this server side
  - elite players are insanely good - they are by definition outliers, so looking for statistical outliers is not in itself a solution
  - game companies are highly incentivized to work with (or at least not antagonize) the elite players (so just throwing them in matches with cheaters is not a solution)
  - the stakes are high both for the devs and their users, so "pretty good" anti-cheat is usually insufficient

You can sum things up by saying that kernel-level anti-cheat DRM is the worst solution, except for all of the other solutions.

I hope to see more discussion on possible solutions and tradeoffs - this is a challenging technical problem whose solution (if there is one) is fairly valuable.

[edit: hopefully fixed the tone, per feedback]

[sunshowers]

While all of what you're saying is true, I think it is worth noting that historically a large chunk of this problem was solved by communities hosting servers. I agree that in the matchmaking era, remote attestation via kernel-level anticheat is the inevitable solution that you converge to after a few iterations.

And yes, servers would often kick out people who were too outside of the general skill level, even if they weren't cheating. As (say) a p80 player, playing against a p99 player feels roughly as bad as playing against a cheater. (But of course the p99 player is doing so honestly.)

[ethbr1]

> historically a large chunk of this problem was solved by communities hosting servers

Yes and no.

I lived through that era too, and there are serious scaling problems: at some point, trying to banhammer griefers with rotating IPs becomes a full time job, and then the public servers turn into a dumpster fire.

[consteval]

Solution: submit them for an account ban.

The games that have the most cheating either:

1. don't do account bans

2. don't limit account creation

You can trivially limit account creation by just charging money for the games.

[ethbr1]

Account bans + community servers don't work either, as the chain of custody for evidence is tainted.

I say consteval was caught cheating on the server I run, and that account should be banned.

Am I, my server's admin, lying?

[consteval]

Probably you need some kind of "court" system. Or maybe if enough dedicated servers say you're cheating, they just ban you.

Yes this is more effort but from the company's perspective they outsource most of the effort to free labor. It can probably be abused if enough admins from different servers band together though.

[sunshowers]

From a labor exploitation point of view it's really hard to argue that that model is better than kernel-level anticheat.

[sunshowers]

I think free to play is where the market has ended up.

I get it, though, I kind of stopped playing competitive games after it became all about the F2P grind. Even cosmetics-only F2P hits a part of my brain that I try keeping in check. I just play single-player and cooperative games now.

[ethbr1]

When one enters the career and/or family stage of life, it doesn't make sense to compete against people who have the time commitments of neither. ;)

[sunshowers]

Yeah, having written that I was thinking about this as well. There's a lot of unpaid labor involved in that model. Maybe, between rootkits and that kind of exploitation of humans, rootkits are the less unjust option.

[ethbr1]

I'd be curious how many anti-cheat rootkit vendors that there are out there, though. It seems like the sort of industry that consolidation to 3-4 larger, more well-funded vendors would be beneficial in terms of security.

Versus everyone rolling their own or using smaller / cheaper solutions.

[jeff_carr]

Not at all correct! Nothing of what was said is true. The actual reality is:

* Microsoft makes piles of money from Gaming * Microsoft got involved with Gaming to damage Linux adoption and corporate support (Sony/Linux/Playstation) * Microsoft spends massive amounts of attention on gaming to lock in the general public to Windows * Microsoft continues to lose to Linux * Microsoft uses cheating to lie about open source being 'something something' cheaters

The fact of the matter is that Microsoft has absolutely no interest in an open source solution to these problems and are using these issues to lie, mislead and spread FUD in some absurd fantasy world where only some superior microsoft driven closed source solution is the only possible way this can be solved. All of that is a complete lie. Nothing more.

A smart linux and free software lawyer would be wise to file a class action lawsuit for discovery documents inside Microsoft where one would undoubtedly find piles of emails between the executives hell bent on doing everything to damage Linux adoption have stupidly wielded this unidentified axe which is actually a -4 cursed boat anchor.

Anyone that tells you that computer security or trust can only be done with proprietary software is lying to you for their own benefit.

[sunshowers]

Sorry, where did Microsoft come in? I'm not sure what Microsoft thinks but I do see both the ups and the downs of remote attestation.

[Gamemaster1379]

> game companies are highly incentivized to work with (or at least not antagonize) the elite players

Actually, this is generally untrue. Companies BELIEVE this but often times, these players are a vocal minority put on pedastal and they often end up making the game worse for the general player base.

[dbrueck]

Sorry for not being more clear, I was referring to the advertising or promotion that comes via the elite players. Take Valorant, for example. Riot Games leveraged their League of Legends user base and gave early access to high-end players and that apparently played a big part in helping its popularity take off. Now it has a robust presence in eSports, again helped by the high-end players.

It's not uncommon now for popular professional streamers to get early access to new features/modes because the game companies know that those players can help build or retain the player base.

[BlueTemplar]

Are they popular because they are the best, or because they are entertaining ?

I wouldn't discount those mediocre (or even outright bad) at the game, but moving huge audiences...

[latexr]

> I'd love to see more curiosity from the HN community on this.

These kinds of sweeping comments are as frequent as they are tiring. There are other comments like yours in this thread and yours is currently at the top. It has nothing to do with a lack of curiosity, you’re simply seeing the contrarian dynamic at play.

https://news.ycombinator.com/item?id=24215601

[dbrueck]

I appreciate the feedback - I've edited the comment to hopefully do better. Thank you for taking the time!

[cmxch]

Rejoinder: Blizzard’s Warden. No bootkit, no invasive system configuration required, even plays nice with “niche/enthusiast” platforms like Linux, doesn’t even care if your keyboard isn’t a bit niche too.

Thought: If they expect a console level of lockdown, why do they bother writing for the PC? If I wanted a $game_console, I’d buy the console.

[dbrueck]

Hmm... isn't Blizzard's main FPS title Overwatch though? Cheating seems pretty common in that game (and there are tons of forum threads where people are complaining about it).

[ethbr1]

Forum threads aren't a great measure of cheating though, given the toxicity and inability of the average gamer to admit "the other player was better than me."

[LinXitoW]

I have about 1500 hours in OW and OW2. I can't recall ever playing with/against a cheater.

[pezezin]

There are tons of forum threads about gamers complaining about every single game in history. Seriously, most gaming forums are incredibly toxic.

[sadeshmukh]

Why isn't server-side anticheat a possible solution? Cheats can spoof inputs purely through visual output as well, meaning there cannot be full trust client-side.

[fireant]

We're mostly talking about FPS here, you've got 2 main cheat categories: aimbots and esps (visibility hacks)

Esps are purely client side, they read actors from game's memory and draw a client side overlay. It's impossible to protect against these on the server. Even if you had perfect culling from the server (didn't send players behind walls for example) you'd still have semitransparent surfaces like foliage and smoke. There are people making good money in PUBG just making enemy textures that are easier to see. You need kernel anticheat to prevent the cheat reading the memory. Also you want to take screenshots periodically and detect overlays.

Aimbots in the olden days could be detected on the server because their movements were instant, precise, unnatural snaps. But these days cheat developers have wisened up. Again the best protection is to prevent the cheat from reading the games memory in the first place, some anticheats go as far as to try to prevent input from any artificial device (so the cheat can't create mouse movement)

There are also movement hacks, but I don't think that these are really common these days. You can detect protect against these on the server side

[Hikikomori]

Just adding that occlusion only works with games that have simple geometry, 90 degree corners, straight corridors, enclosed areas and no large difference in elevation. So its useful in games like CS or Valorant, but will not work well in open games like Battlefield or Escape from Tarkov.

There's also DMA based cheats that will read memory with another computer which can then output an ESP overlay to a HDMI or DP merge box that will show both the game and overlay on your monitor. They can also do aimbot by adding mouse inputs to a device you connect your mouse.

Websites sell DMA cards and these other devices together.

[burnished]

Not an expert but I've done a little reading and basically the combination of real time actions and a network makes it intractable, you end up just having to trust the client on some things (or having to make trade offs like a client potentially not having the information needed to display the game state to the player, or choppy/unresponsive gameplay as a function of latency).

[FactKnower69]

>some things

Any specific examples? I hear this said all the time and it's almost never true.

Movement, for example: many decide to just let clients be fully authoritative over their positions and then act shocked when teleport hacks drop. Just keep track of the player's max move speed server-side, continually validate, and flag if they consistently move faster than is possible according to the server. No one is ever saying you have to validate inputs server-side in lock step with zero client-side prediction whatsoever and enforce 200ms of input lag for all players.

[lifeformed]

It's not teleporting that's hard to deal with, it's aimbots and wall hacks. You have to trust the client with enemy position information that it shouldn't be able to see yet, and trust their shot position inputs.

Also, constantly flying around and teleporting is easy to catch, but using it in small bursts is very powerful and harder to catch.

[tga_d]

>You have to trust the client with enemy position information that it shouldn't be able to see yet

That seems like something that would be solvable with location-style differential privacy. Report a number of plausible locations to the client small enough that it can efficiently anticipate them all, but large enough to prevent being able to auto-aim or wall hack. Run some bots or actual player movements recorded from other matches, originating from roughly the same point where you last saw the real opponent.

>constantly flying around and teleporting is easy to catch, but using it in small bursts is very powerful and harder to catch

Even small violations of continuity seem like they'd be observable server-side, no? I've not studied this, but presumably clients must be constantly phoning home with their position.

[musjleman]

> That seems like something that would be solvable with location-style differential privacy. Report a number of plausible locations to the client small enough that it can efficiently anticipate them all, but large enough to prevent being able to auto-aim or wall hack. Run some bots or actual player movements recorded from other matches, originating from roughly the same point where you last saw the real opponent.

Has already been done in COD: Warzone. Varying levels of success, cheat developers end up heuristically eliminating fake players.

> Even small violations of continuity seem like they'd be observable server-side, no? I've not studied this, but presumably clients must be constantly phoning home with their position.

This issue usually is game/game-engine dependent and is achieved either by exploiting bugs or manipulating lag compensation. Not exactly a very common thing.

[lifeformed]

> Even small violations of continuity seem like they'd be observable server-side, no? I've not studied this, but presumably clients must be constantly phoning home with their position.

Jumps in position are not always illegal: network issues, quirks from physics-based forces, glitches in the game, are all very common and can all cause unexpected positions. Differentiating from bannable offenses is not easy. Yes, there's always heuristics you can use to narrow down possible issues, but you have a limited CPU budget: You need to be running multiple instances per machine, each updating 60 times a second, serving dozens of players, sending and receiving constant updates to and from all players 30-60 times a second, while simulating physics, large worlds, complex player states, and synchronizing the states of thousands of objects. It's tricky to get everything right and performant. And people will get extremely mad if you make a false positive.

> That seems like something that would be solvable with location-style differential privacy. Report a number of plausible locations to the client small enough that it can efficiently anticipate them all, but large enough to prevent being able to auto-aim or wall hack. Run some bots or actual player movements recorded from other matches, originating from roughly the same point where you last saw the real opponent.

But what is the client suppose to do when actually seeing the real position? At someone the waveform needs to collapse and reveal the real location. The only way to make the fake locations indistinguishable from the real ones is to make them a real enemy player from the client's point of view. But then you stumble across all these fake enemies that don't do anything? You could place them in unreachable positions so normal players wouldn't ever find them. But then the heuristics for checking if a client "knows" about the position is still quite fuzzy. Also, visuals aren't the only giveaway of an enemy location. Audio is also location based. Playing fake audio would be detrimental to normal players' experiences.

Having said that, the unreachable-fake-player technique is not bad, it can cut out some low hanging fruit. But it's only part of the equation of a robust anti-cheat solution. It's complex to implement and only gets you some cheaters.

[burnished]

A simple example is clock timing in chess, you have to trust the client about when it received and when it sent if you want to avoid treating everyone as a cheater and penalizing genuine latency.

If I remember right an anecdote from someone in the trenches was along these lines for a more complicated real time game, though I think the peer comments have the more typical types of problems. I looked for the thread I was reading this in but came up dry, sorry!

[ukd1]

Oh it's a solution, it's just worse than kernel-level - as it's much easier to bypass.

[phire]

A good anti-cheat solution needs both client side and server side components, they complement each other.

You also need active human involvement, both as moderators and evolving the anti-cheat technology over time.

[nox101]

Do you think it will escalate to the point that client side checks will be worthless? Say in 5yrs I can let an AI watch the screen and control the mouse and keyboard. From the rootkitted computer, it can't tell I used an external AI to control the USB keyboard and mouse.

[zerd]

In 5 years -> hobby project 3 years ago: https://www.youtube.com/watch?v=LXA7zXVz8A4

[bitsandboots]

Perhaps true of competitive games. But I find anticheat mainly exists to protect microtransaction games. And generally those games aren't worth playing, so having the little badge on steam can help avoid wastes of time.

[127]

Just use local servers and player validation signatures. Faceless matchmaking is bullshit. Local communities win. Don't mix e-sports with casual game-play. Just like you don't need a security detail for the average person, you don't need invasive anti-cheat for the average gamer.

[FactKnower69]

>no, you can't completely solve this server side

This is what every dev who can't be bothered to implement relevancy filters says when their server broadcasts the locations of every hidden player to every other player every tick and wallhacks drop a week later

Exactly what can't be fixed server side? Are you just talking about aimbots and other situations where script kiddies can trivially author bots that generate optimal inputs? Because at a certain point that's more a problem with shitty, boring game design that got stale 20 years ago; if the top of your game's execution ceiling is "can the player click on heads perfectly" you have bigger problems

[dbrueck]

Relevancy filtering is more for network traffic optimization, it doesn't really help with cheating in most cases. In a FPS, for example, the actors the cheater most wants to know about are almost always also network relevant.

But taking a step back, for fast games (like an FPS), the latency requirements drive you to send semi-secret info to the client (like the positions of other players), and so that's where things start to break down. But the traffic in the other direction is a problem too, as you have all of the scenarios in which the messages to the server (e.g. aim info, timing of weapon of firing) can be spoofed or engineered.

The motivation for the client-side anti-cheat systems is to extend as far as possible the envelope of what is considered trustworthy - i.e. if they can't solve the latency problem, then they try to make the client more trusted.

It's impossible to completely solve the problem, so it's about finding a solution that solves as much of the problem as possible. Unfortunately the main thing going for kernel anti-cheat is that most users don't care that they have to let someone root their machines to play a game, though the tide would likely turn if there were a high publicity exploit.

[lifeformed]

"All cheats can be trivially solved server side, as long as I exclude all games I don't like, which are also the games where the problem is hardest to solve and most relevant to the discussion."

[Hikikomori]

Server side can not do anything about ESP or aimbot as they rely only in information that the server must provide the to the client. ESP can be curbed somewhat by obfuscating objects not in their view, but how effective this is depends very much on map geometry as the server must send it at some point. It works okay in games like CS/Valorant (that already has it implemented for years) but does basically nothing in Battlefield/Apex/Escape from Tarkov as they have very open maps. Aimbot can be configured to be pretty much indistinguishable from the best players.

[talldayo]

> I'd love to see more curiosity from the HN community on this.

I'd love to see more curiosity from developers - the disappointment is mutual. Instead of attempting to systematically stop all forms of cheating through innovative or competitive methods, it would appear the industry is converging on dangerous half-measures and excusing it with evidence from a clearly failing system.

What should we, the users, expect? Perfect, cheat-free software that surveils us endlessly, or "good enough" security that lets users decide for themselves which servers are suitable? Let me cast my vote, and I know which ideal I consider realistic and attainable.

[Levitating]

> I'd love to see more curiosity from developers

Developers spent millions on Anti-Cheat. It's why entire products like EasyAntiCheat and BattleEye exists.

Valve spent a LOT of time and effort on VACNet, a server side machine learning based Anti-Cheat primarily trained only on CS:GO verdicts and it was awful still.

Developers know the common methods used by cheaters. That includes exploiting known vulnerable kernel drivers to run code in the kernel. The only way to monitor for this is to utilize a kernel module loaded before that of the cheater. That's why the current state of Anti-Cheat is the way it is.

The developers of various anti-cheats like Vanguard have been very transparent about this.[1]

[1]: https://www.leagueoflegends.com/en-us/news/dev/dev-null-anti...