[evervevdww221]

I used to work on (the second generation) cloud gaming. It's very difficult to be profitable. Most people would think that the challenge is latency, streaming quality. But the real challenge is cost management. The cost can be easily as high as $2 per hour. At this rate, the monthly cost will buy you a decent computer or a console. It doesn't make sense.

Plus, to make cloud gaming work, you have to own everything. From the video encoding hardware, operating system, virtualization, game content, distribution channel, cloud ...

For example, we tried to hack windows to support existing windows games. It was very difficult. Windows isn't a multi-user system. There is no proper user isolation. We have to monitor the hard drive to see what files are touched by games, and try to resolve conflicts caused by different users using the same machine. It's very hacky. An easier way is just discarding the VM and refreshing the hard drive, but it will result in long loading time of several minutes, very high cost and poor experience.

And then there is the game store problem. Game publishers won't share revenue. You basically sell the games at the same price and also charge users for the cloud gaming platform for a compromised experience. doesn't make sense.

There is only one company that owns the whole stack, Microsoft. They own the OS and APIs (so that includes all the drivers), virtualization, games (XBox store), data centers (Azure).

A few years ago, YC invested in a cloud app streaming company (they kinda used the same technology we used.). I was surprised. After working on it for a few years, I would not invest if I were an investor.

Google's stadia seems to be smarter. As it tries to avoid some of the dead-ends we went to, especially trying to hack windows to make it cloud gaming friendly. Google chose to use Linux and develop games tailored for cloud gaming. I think that's a better choice, but will face content issues.

[akvadrako]

How does it make sense that a computer at home, with 5% utilization and retail power prices, can be less per hour than a computer with 50% utilization and wholesale power?

Even if all you do is host single-user machines with retail games installed and some kind of imaging solution, it seems like it should pay off.

[evervevdww221]

$2 per hour per machine is not a made-up number, just check how much Amazon charges for a GPU vm.

Again, we were a small company, we had to use other cloud providers. I don't know why amazon or alike are so expensive. We tried colocation too, not any cheaper to be honest.

Also, when buying a computer or a console, you don't pay by usage. you pay one time to own the hardware and the remaining 5 year usage is free, plus power bills no one really cares. 5% utilization times 5 year is still some decent hours, the cost is not necessarily more than a cloud vm.

For a cloud vm, you pay by usage. The more you use, the more you pay, the overall payment is not capped. Eventually the cost will surplus that of the hardware. And I'm saying "Eventually" is actually a month.

[mantap]

Amazon is a bad fit for this use case. Amazon's value proposition is: "you are making money, pay us a big chunk of it and we will help you scale up with less effort and fewer staff.

It doesn't seem surprising to me that making a cloud gaming service would necessitate assembling your own servers. Games just have different hardware requirements to everything else, it's well known that "pro" graphics cards are not meant for gaming.

[aeturnum]

Not Op, but they specifically said the using a colocation (i.e. setting up your own servers) was not helpful in keeping down costs.

[mantap]

I know but I find it hard to believe that Colo provides no benefit. I'm guessing a large part of their AWS bill would be bandwidth since this kind of a pathological use case for AWS (streaming individual video with no possibility to use CDN).

[Tsarbomb]

> And I'm saying "Eventually" is actually a month.

I was the technical lead on a project where I warned the product owner that implementing a certain feature the way they wanted in AWS would have astronomical costs. I was told "let me worry about the money".

A month later we were told to scale down our ECS and EMR usage as the bill was astronomical.

[akvadrako]

I assumed you were not using Amazon because it's obviously a bad choice for this use case. $2/hour is $87k over 5 years — not surprising you can't make money.

One can rent a rack for about $3k a year which comfortably fits 20 servers. Obviously, colocation is much cheaper.

[HeWhoLurksLate]

You've also gotta get servers to put in there, and in a 42u rack with, say 9 4u servers, a 2u managed switch, and now you have 4 units left to handle power distribution, power backups, any kind of external remote management (say an out-of-band KVM, or a second network for IPMI traffic, or anything else).

ALSO, all those servers? Say the fancy, high-speed switch you need to route all that traffic is $2000.

You wind up needing spend $5000 on auxiliary equipment and installation. This includes IE power cables, network cables, Velcro, and that darn cable you forgot you needed.

Now we're up to $10,000 including the colo costs, and we haven't even gotten to servers yet.

We need 9 4u-tall servers. The reason for the 4u height is because that's the size you want to be the most space-efficient with your full-height GPU's.

You'll want good base servers to slap your graphics cards into. I'm a little out of the loop on the latest and greatest in the server world, but we'll assume it's around $7,500 for a fairly moderate AMD EPYC system (EPYC because A] they wind up being cheaper than their Intel counterparts and B] have many more PCIE lanes).

9 $7,500 servers is $67,500 without any GPU's.

You don't want nVidia's consumer GPU's, because they're hostile to virtualization, so it's either AMD's consumer or enterprise stuff Or nVidia's enterprise stuff.

For GPU's you want to use, you'll be paying at least ~$600 per, and there's around 8-10 slots per server. $600 * 8 * 9 = $43,200.

I'm sure I've missed stuff- haven't included data storage, for one- but we're already at $120,700 and you'll probably want some new GPU's in a couple of years, and the total cost of servers over their lifespan winds up being around double what the initial cost was.

-Summer Glau

[akvadrako]

I wasn’t suggesting doing it this way. I said to use the equivalent of having a system at home: no virtualization, consumer GPUs. That way it’s an apples to apples comparison - you just get higher utilization and cheaper power. The cost of the switch and rack are minor when divided over 20 systems.

[hrgiger]

AMD consumer GPUs doesnt support SR-IOV probably something like this https://www.amd.com/en/graphics/workstation-virtual-graphics

[perennate]

I think the costs are higher for local gaming than cloud gaming. But the user views the costs differently -- with local gaming, they own a GPU after buying it, and the electricity charges are not really apparent; but with cloud gaming, they pay a clear monthly fee. So that fee can't be too high or users won't be interested. And on top of all this, you have to add the tradeoff of the latency and bandwidth usage versus local gaming where these are not a concern.

[evervevdww221]

one more detail regarding cloud utilization. It might not be what you think it is.

Game playing activity peaks roughly at the same time during a day, mainly in the evening. Basically you need to launch one vm per user during that time. And those vms will be idle during the day time. Sure, due to time differences, those vms can be used by other people from a different time zone. But the further the data center is, the poorer the experience will be.

[jzl]

On top of all that, Google has said that they plan to roll out over 7500 edge computing clusters just for Stadia, all for tackling the latency problem. (Not clear if those clusters can be put in existing Google facilities or not.) Having so many edge clusters runs counter to the ability to amortize costs by ensuring fully subscribed hardware usage. Each edge cluster will be as costly as the required peak usage for each location. And I imagine Stadia will have a very distinct pattern of peaks and troughs of usage relative to local time.

[perennate]

Sounds like we need a startup that offers mobile flying datacenters!

[rstupek]

Because to run the service you have to pay for that 50% utilized computer and associated infrastructure where you don't when its someone else's?

[perennate]

Hm, I think I see your point.

The biggest benefit of cloud gaming to companies is that more people can access more video games. Under other cloud gaming services, the developers benefit more than the cloud gaming platform. But Stadia's closed platform means Stadia can take a cut of the real profits.

[dmix]

> The cost can be easily as high as $2 per hour. At this rate, the monthly cost will buy you a decent compute

This goes against everything I learned in business finance/accounting class (that was oblivious to software/internet) and all modern SaaS business models.