[Traster]

The average person on twitter isn't a software engineer. They won't know what python is. They don't know what an algorithm is. So let's be clear about what the baseline is here. There's no point talking to them about how you distribute queries using micro-services or whaever bullshit twitter's engineering team bought into this season.

This idea that "the algorithm" is something you can just "publish" is a pernicious lie told by people like Musk - who knows it isn't true - to the general public who don't know better. The "algorithm" in reality is probably farcical calls to cusotm APIs that no current employees understand well enough to modify which is why Twitter hasn't changed anything in coming up to a decade - which is when all the engineering talent left.

[farias0]

I believe all the public wants is a report with an overview of the mechanisms that dictate how and in which order tweets are shown to different users. Even if no current Twitter employee knows the system well enough to write it, they can still create a taskforce to do so. Sure, 3rd party APIs and machine learning stuff may obfuscate part of the system, but I'm sure the best they can do is good enough for starters.

[cscurmudgeon]

This is a bad take. Just because it is a stupid unaccountable heap of shit now doesn't mean it has to remain that way in the future.

[Traster]

I'm not talking about whether it's accountable or not, I'm saying that to actually share the algorithm you're basically saying that you're going to open source Twitter's entire code base. Oh and when you do that the average person will be no better off because they don't read code anyway. And when engineers read through the code it's going to like "Where did you get all these different variable values from?" and the answer isn't "We came up with a method for valuing tweets from first principles", it's going to be "We showed 7 billion people tweet X and 7 billion people tweet Y and tweet X caused 5% more people to engage so we tweaked this value".

And sure, you can say "Well that's a bad way of designing the algorithm" but then what you're really saying is that you don't want to open source the algorithm at all, you want to re-write the algorithm to satisfy your sense of how the world should work with no evidence it'll actually work.

[cscurmudgeon]

> , it's going to be "We showed 7 billion people tweet X and 7 billion people tweet Y and tweet X caused 5% more people to engage so we tweaked this value".

There is an entire new subfield of ML that is tackling this problem. There are now conferences dedicated to this topic. It is not an easy problem, but it is not impossible.

There are hundreds of researchers working on fairness, interpretability, trust and explainability in ML and a lot of them are working on models much much bigger than what Twitter's feed might involve.

This is a good starting point:

[1] https://www.fatml.org

> And sure, you can say "Well that's a bad way of designing the algorithm" but then what you're really saying is that you don't want to open source the algorithm at all, you want to re-write the algorithm to satisfy your sense of how the world should work with no evidence it'll actually work.

You can still open source multiple steaming piles of shit and then let the community improve that so that it is more widely understandable and trusted. See [1] again.