Hacker News new | ask | show | jobs
by joepie91_ 3157 days ago
> Caddy is a server which is equivalent of nginx part in Captain. It's orange and apple. Arguably, I can replace the nginx part with Caddy. But captain has more to offer, like building pipeline and scaling.

The problem is that these aren't generalizable problems. Scaling requirements are going to vary depending on the project, also because of the state management issue I mentioned. This, in turn, means that the only projects you can scale with an out-of-the-box approach like this, are those that don't need to scale (horizontally) in the first place. You therefore get little to no added real-world value from these 'scaling' features.

I should point out that this isn't a problem just with your project; I've noticed a general tendency around Hacker News circles to overengineer for 'scaling' (which usually refers specifically to horizontal scaling), trying to invent magic scaling solutions that work for everybody (which don't exist), and completely ignoring that distributed systems are hard and you usually want to scale vertically first.

This problem is compounded by the fact that most applications don't actually need to scale horizontally, since they never get big enough to require it. This means that reviews of such 'magical scaling' tools are almost universally positive, because everybody is reviewing them based on the claimed benefits, not based on real-world experience with scaling requirements. This results in a feedback loop of recommendations for completely ineffective scaling strategies by people who never actually needed them.

> Great point about customization! I'll add more hooks in the future to account for non-standard nginx configurations.

While this would seem to improve the project when viewed in isolation, it's important to realize that you're very likely to fall into the trap of non-generalizable problems.

A lot of monolithic frameworks actually have this issue; they try to provide generic implementations for problems that just aren't generalizable (because they vary too much by project, such as "content management"), and in the process they end up providing hundreds or thousands of different configuration options and hooks... to a point of complexity where it would've been easier and more reliable to just not use the framework at all. A typical example of this are general-purpose CMSes, which are almost inevitably unreliable, messy and insecure.

There's a serious risk of that happening here as well, if you start adding support for more customized setups; at some point, you're going to reach the stage where the complexity of your configuration options exceeds that of the tools you're using behind the scenes (nginx, Docker, etc.). I suppose that'd be a form of the inner-platform effect, too.

I don't mean to discourage you here, but your current approach is very likely to result in something that tries to do a lot of things and none of them particularly well; not because you're implementing it wrong, but because this particular set of features just isn't generalizable as such. You'd probably get better results by providing a collection of tools (and documentation!) for different tasks, that 1) can be used independently, and 2) don't require full control over the infrastructure configuration.

> Finally, Captain doesn't support persistent data, i.e. database. Therefore, scaling up apps is much easier.

I don't agree that "scaling up apps is much easier" - state isn't something that's optional, pretty much any real-world project (especially those that have to scale beyond one instance!) has to persist state of some sort. There's only the choice of whether the tooling supports it, or doesn't. I'd consider it dangerous to present something as magically scalable without addressing this issue; it's bound to result in people having state consistency issues.