[morgante]

> Running on rented hardware is the equivalent of a traditional product company renting all of their factories.

Correct.

Traditional product companies very rarely own all their own factories.

Do you honestly think Netflix management is incompetent? They've run the numbers on this countless times. They have way more insight into their spend and operations than a random anonymous internet commentator.

[x0x0]

Netflix is a classic case of very bursty demand when they eg regularly re-encode all video, or even just see huge audience swings when people get home from work and turn the tv on.

Most companies at even 1k servers will get enormous cost savings from O&O.

A former employer, then in the 600-ish box range, costed out moving to ec2. Our ops actually brought up our internal hadoop cluster on ec2 as well as ran parallel workloads for a week. Amazon at the time offered price deals to attempt to get them as a marquee customer, as well as a bunch of engineering time. Even so, ec2 cost 3-4x as much for 1/3 the throughput. ec2's prices have since fallen, but so has the price of bare hardware.

People also overestimate the cost of hands-on; this company contracted for 4 hours / week with a local person. If you get serious about puppet/chef you can successfully run lots of servers in a remote datacenter.

[keepper]

Netflix runs their most expensive part (pushing those terrabits of traffic ), with their own hardware and network. https://openconnect.netflix.com/en/

so, you were saying?

Or do you think facebook,google,aol,tripadvisor and pretty much anyone beside HotNewStartup are incompetent?

[mynameisvlad]

More than likely so that the video feed is as close to you as possible, rather than only because of cost efficiency.

The video is the most important part of their product, so keeping it at your ISP or a local exchange not only lowers latency, but most likely also gives better throughput since it only has to travel through your ISP's network or local exchange links without having to go through the rest of the internet backbone.

[keepper]

Its not "more thank likely", it is the real reason. CDN's could not keep with their volume AS WELL as pricing structure.

Latency ( 30ms vs 300ms) doesn't matter much in video playback, only throughput. Once you hit play, it isn't bi-directional.

But you're just nitpicking. This IS their core business, and they realized its "too core" to give to someone else.

So yes, you do "outgrow the cloud", and well, part of "outgrowing" can be cost structure. Gitlab offers a free service, their main competitor, github, runs their own datacenter. Why do you think that is?

[mynameisvlad]

Right, but that alone shows that the reason they didn't go for the cloud for those has nothing to do with the capabilities of it, just that the locations are not close enough to their userbase to make sense.

Put another way, if there was a cloud DC in each major city and Netflix hadn't already started using their own edge nodes, then there's nothing stopping them from spinning them up in each cloud DC as opposed to their own hardware. There's nothing inherently special about their setup except location.

Nothing about this situation says that Netflix "realized its "too core" to give to someone else." I don't quite understand how you're making that leap. Instead, they had a very specific requirement (that most companies don't have, mind you) that current cloud hosts can't provide. Nothing else.

[StreamBright]

Even Netflix uses AWS for the autoscaling enabled workloads.

Besides, here is a list of companies using AWS:

https://aws.amazon.com/solutions/case-studies/

You think Google and Microsoft is incompetent when they offer cloud services to their customers? There are a lot of companies that do know want to own any datacenter related infrastructure because it is irrelevant to their business and there is no on-board expertise. While there are companies that can afford to hire staff for running large DC operations. I thought this is pretty obvious to everybody.

[morgante]

I know Netflix runs their own edge nodes, which is precisely why OP sounds incompetent and uninformed. Netflix has spent lots of time and effort into analyzing and building the architecture which is optimized for them.

> Or do you think facebook,google,aol,tripadvisor and pretty much anyone beside HotNewStartup are incompetent?

Far from it, and that's precisely my point. At Facebook scale, it certainly makes sense to run your own data centers. At HotNewStartup, it obviously doesn't. Competence is finding the solution which works best for your scale and organization, not following some generic rule.

[keepper]

I bet to wager Netflix and Snapchat are at Facebook scale. Certainly $500M/ year in cloud commitments for the next 4 years seem to indicate so.

[morgante]

I'm not sure, but I am sure that someone internally is doing those calculations.