[cubes]

Hard agree.

History is littered with dead startups that designed for scale before they had enough usage to justify it. Within reason, having users knock your site over resulting in failures like the Twitter Fail Whale is a good problem to have.

With that said, you need to be prepared to scale up quickly once you have this problem. There's a reason Facebook counts its users in the billions, and Friendster is a footnote in internet history.

[s1k3s]

Yes, but.. I am a software architect. ~90% of the job offers I get are from CEOs/CTOs who want me to join their small company to help them refactor because they can’t grow anymore. Tech debt kills as well.

[dgb23]

Is that a "but" or is that how things should work? You solve a problem when you have it or can reasonably predict it (in technical/engineering terms). I also think it's sensible to look for experts when you need them right?

[colechristensen]

I don't know I've been places where the difficulty scaling and failure to do it promptly posed an existential threat. As in "this system is poorly designed and needs to be rewritten so we're going to pause new features for six months to do it", you can't always just magically find an expert to solve your problems in a timely manner, and feeding the technical debt monster can easily kill startups.

[icedchai]

For every company that has this problem, another several never made it far enough for it to matter.

[noodle]

If they have the money to hire you and they have pent-up growth that refactoring will unlock, that really sounds like close to an ideal path for a startup to take. They're literally paying down their tech debt by paying to hire you. Maybe ideal would be a little earlier so it doesn't block growth, but yeah. In my experience way more startups have died from premature optimization

[xboxnolifes]

If this pattern is successful, it's not killing them.

[raffraffraff]

So how do you square they circle "Don't design to scale but scale up quickly when you need to"? You can't rewrite your stack just because you took on a new customer that is suddenly killing you.

I don't think that it's stupidly hard to get basic things set up for scaling at the start. These don't apply to everyone but:

- use a cloud provider, they let you grow

- write infrastructure as code (terraform, pulumi, cdk), it's not that hard and it makes building your 2nd, 3rd data center easier

- for the love of God think for 10 minutes about a scalable naming convention for data centers, infrastructure components, subdomains, IP ranges etc. This type of stuff causes amazing headaches if you get it wrong, and all you have to do is consider it at the start. Many AWS resources can't be renamed non-destructively, and are globally namespaced. Be aware of this. Don't paint yourself into a really stupid corner.

- even if you're not doing microservices, consider containers and an orchestrator that has scalability baked in (like Kubernetes). If you use a managed Kubernetes with managed node groups or fargate, you can basically forget about compute infrastructure from this point on

- deploy Prometheus, Grafana, Loki etc day one and build basic dashboards. With Kubernetes, you can get this installed quickly and within a day you'll at least be able to graph request/error count and parse logs.

- deploy an environment for developers to deploy to and use the product (even load test it)

The development cycle is now set up for building what you need to build. You can easily find developers at any level who have experience with containers, Kubernetes, Grafana so they can onboard quickly, be productive and troubleshoot their own stuff. They can refactor the monolith without having to invent service discovery, load balancing, ingress etc.

My personal experience with Kubernetes began at a start-up with a tiny team, none of whom had prior experience with it. We had started building a single monolithic app. But within a short time we figured out how to make a helm chart, add ingress and boom, we were "online". Every time someone was about to spend time and effort inventing something, we would quickly scan through the Kubernetes docs and CKAD material to see if we could avoid having to build it at all. We leverage cloud stuff too, where it was cheap and easy and scalable (like AWS SQS/SES/SNS/CodeArtifact/ECR/Parameter Store)

[iovrthoughtthis]

i didn't see the words "it depends" anywhere in this

thats the whole point of engineering, picking/building the right solutions for the problems and context

some stuff just needs a vps, a php file and an sqlite db and it will scale

other stuff might need container orchestration or anywhere in-between (or beyond). it should depend on the context though and we should be prepared to change strategy as needed

[raffraffraff]

I did say "These don't apply to everyone but:" and then gave my generic list of stuff that works for a lot of people.

I get that some people can just run a single go binary or whatever. The ones that don't, need to set themselves up to grow easily when they need to. Having some sort of basic plan or framework that doesn't take more than a week to set up is due diligence, and hiring (when your need to) will be much easier.

[bryanrasmussen]

>With that said, you need to be prepared to scale up quickly once you have this problem.

See I interpret that to mean that you should design for scale, but not implement or invest in scale until needed.

[travisgriggs]

Is one of the problems though is that the early audiences appreciate this kind of over-engineering? If your investors/buy-in folks are being shown early demos and you mention "and this thing is gonna scale seamlessly up to 1,000,000 users because of tech words A, B, and C" then they get excited. They don't understand A, B, or C. But what they just heard is that you're planning for Big. And that stokes their aspirations.

[roflyear]

Facebook is a great example. They did tons of things not traditionally known as highly scalable. Like one repo with the project coded in php which was largely a monolith.

[cutler]

And an imperative codebase which looked like something out of Matt's Script Archive. I often wonder whether a startup today could pull-off something similar.

[jasonkester]

Absolutely they could. And they should, because doing so will get you launched quickly.

Instead, todays shops all start out with javascript front ends and 40 layers of backend complexity to accomplish the same thing.

[cutler]

I'm not sure. The bar for UI has been raised significantly since Facebook launched and mobile has changed the game to such an extent that you need a presence on 3 platforms out of the door (iOS, Android & web) to even stand a chance.

[roflyear]

Not entirely true for many apps. The app I work on does not need a working mobile site, because we don't think our consumers are going to use it on mobile.

However, we do very much care about our UI - and I think react at this point helps with that. But I don't think React complicates the stack too much. It may even make things easier, tbh - I don't have to test pages on backend tech, just endpoints.

[cutler]

React easier compared with what - imperative PHP4 at Facebook in 2004? I'd have to disagree. React state management is much heavier than returning HTML from an embedded PHP template.

[roflyear]

I believe so. But, most startups like to use insane tech for fun, or something - though in my experience all it does is kill productivity.