[John23832]

Even at Netflix RxJava is not fully accepted. When I was there I wrote a service which called many downstreams. It orchestrated test user creation. I used RxJava because it was being pushed by the platform team, and coupled well with our internal GRPC api handlers.

I shared with the team and people complained about the complexity… so I ended up ripping it out. What people wanted was a test user creation service that worked well and could be maintained… not one that was the most efficient.

[criddell]

Looking back, do you think choosing practical reasons over efficiency was the right call?

[John23832]

Yes. Code is read (and debugged) way more than it is written.

I was using RxJava initially because it was being pushed by the platform team and coupled pretty well with the GRPC interfaces we used internally. However, the rest of my team couldn't/didn't want to wrap their minds around the concurrently functional tasking that it brought (which was understandable).

I didn't want the code to have a bus factor of 1 (me).

[conceptme]

Is there any reason they don't adopt kotlin? It seems a lot simpeler with suspended functions and coroutines?Instead of RXJava.

[John23832]

In growth we had a failed experience with groovy… but Netflix wide? I think they just wanted to stick with basic Java.

[doctorpangloss]

Adopting RxJava means fixing all the bugs in everything up front. That is why it feels complex. In my opinion and in the opinion of other adopters, it is a more expressive way to define correct behavior.

This is essentially the same tension between static and dynamic typing.

No matter what, all bug free code is complex. Correct code can be written in any paradigm.

But productful code ships with bugs.

Does test user creation need to be bug free? I don’t know. It feels like something where productful test user creation can ship with bugs, even if in reality your test code harnesses should not be buggy.

You can step even further back and ask: when you have essentially unlimited resources, why ship buggy code at all? Netflix has unlimited capital, and it isn’t in a hurry anywhere. Engineers should never choose productful over bug-free. You learn nothing (Netflix’s product problem will never resemble your startup’s product problem) and you forfeit the opportunity to learn (ReactiveX is a good, expressive paradigm).

This is a startup forum, is this true for startups? IMO, yes. Your startup is going to fail. You will not learn valuable product lessons from the journey. I mean people certainly pretend and say they do, but even in the situation of success, Brian Chesky still hasn’t become Walt Disney last I checked. But once you have enough money to be employed for a year, as an engineer, you know, fucking learn how to do something the right way, bug free and performantly.

So I agree your coworkers are fundamentally making an error.

[John23832]

> Adopting RxJava means fixing all the bugs in everything up front. That is why it feels complex.

I disagree with this. It mentally changes the thought processes when debugging, which was the biggest issue that was brought up.

> Does test user creation need to be bug free? I don’t know. It feels like something where productful test user creation can ship with bugs, even if in reality your test code harnesses should not be buggy.

It should be bug free at Netflix scale. My service created more customers daily than Netflix signed up monthly. It also sat in the middle of Growth. People use it to gate deployments. It needs to work flawlessly (or at least consistently).

[jakewins]

This was part of some “testing in production” type scheme?

Are there any talks or blog posts on how y’all wired this up, lessons learned etc?

[John23832]

No this all happened in preprod. Every team in the growth space ran their own tests across the various permutations of users (country, payment type, plan, ... at the time free trial, feature flags) to determine if their code was valid and worked to go to production. Each team continuously deployed independently. Given that we also were continuously integrating as well... you get a lot of customers created.

Sorry no talks.

[brabel]

I tried using an API based on RxJava to implement some async code... It was buggy as hell, as using the API correctly was nearly impossible for me - that's the opposite of a good API which makes it very hard to use wrong. I am in the process of replacing all that shitty code with Virtual Threads, and all bugs just immediately disappear as the code is so much more straightforward to analyse.