[Kwpolska]

> using an (often expensive) feature flags as a service platform

I have no idea why anyone would actually do that in real life. Feature flags are something so trivial that you can implement them from scratch in a few hours, tops — and that includes some management UI.

[jitl]

Often these 3rd party offerings are feature flags PLUS experimentation with user segmenting. Depending on the style of software you build, this can be extremely valuable; it’s very popular in the SaaS market for a reason.

Early on at Notion we used simple percent rollout in Redis, then we built our own flag & experimentation system, but as our needs got more complex we ended up switching to a 3rd party rather than dedicating a team to keep building out the internal system.

We will probably hit a scale in a few years where it makes sense to bring this back in house but there’s certainly a sweet spot for the 3rd party version between the 50-500 engineer mark for SaaS companies.

[boulos]

That's a reasonable path! You probably learned to appreciate and value the complexity, but you wouldn't have from the start. Which service do you use?

[simonw]

That path sounds very sensible to me.

[cogman10]

Happens when you do the flags wrong :)

We have a FF as a service platform and a big "value add" is that we can turn on and off features at the client level with it.

But, unfortunately, it's both not the only mechanism for this and it is also being used for actual feature flags and not just client specific configuration.

I'm personally a MUCH bigger fan of putting feature flags in a configuration file that you deploy either with the application or though some mechanism like kubernetes configs. It's faster, easier to manage, and really easy to quickly answer the question "What's turned on, why, and for how long". Because, a core part of managing feature flags is deleting them and the old code path once you are confident things are "right".

The biggest headache of our FF-ws is that's really not clear and we OFTEN end up with years old feature flags that are on with the old code path still existing even though it's unexercised.

[hirsin]

You'll still be building management UI over their system (it doesn't understand or validate actor types, tenants, etc, so you have to do that.).

But at high throughput, you might want something with dedicated professional love. Ten thousand feature flags, being checked at around 2 (or 200) million RPS from multiple deployments... I don't want to be the team with that as their side project. And once you're talking a team of three to six engineers to build all this out, maybe it makes sense to just buy something for half a million a year. Assuming it can actually fit your model.

[Spivak]

But it's not a side-project in most implementations it's part of the app itself.

[jasonkarns]

Side project means it’s not that team’s primary focus.

[fizx]

The scale is easy in practice, cause you outsource to a CDN. But everything takes time and has opportunity cost.

[hirsin]

Maybe we've worked with different FF systems, but anything that involves a call more expensive than an RPC would be lethal to request latency. Calling out to a CDN forty five times per inbound request would be... Infeasible

[fizx]

A background thread long polls the CDN, updating a local hashmap on change.

[fizx]

If I was a bootstrapped startup, I'd do a json file and then when I've outgrown, I'd hand write something that long-polls a CDN for updates, with a tiny rails or react app behind the CDN.

But these approaches are insane for companies above a certain size, where individuals are being hired and fired regularly, security matters, and feature flags are in the critical path of revenue.

Last time I looked at LaunchDarkly Enterprise licensing, it started at $50k/year, and included SAML.

Now that sounds like a lot, but if you're well past the startup stage, you need a tiny team to manage your homegrown platform. Maybe you have other things for them to do as well, but you probably need 3 people devoting at least 25% of their time to this, in order to maintain. So that's at least $175k/year in the USA, and if your company is growing, then probably the opportunity cost is higher.

[ozim]

Add to that ideally feature flags should be removed after feature is released. Ideally also you shouldn’t have more than handful of feature flags.

Permanent per customer configuration is not a feature flag. Also best would be not to have too many per customer configurations.

[Supermancho]

Feature flags are ofter initially for feature and later left in as dependency flags. Even within a large organization, individual components and services by other teams will have outages.

[ozim]

That sounds like bad engineering.

Having all kinds of flags makes system prone to misconfigurations.

As amount grows you get flags depending on other flags etc.

Gets insane rather quickly. Unless someone purges flags relentlessly.