[jamesblonde]

I gave a talk at PyData Berlin on how to build your own TikTok recommendation algorithm. The TikTok personalized recommendation engine is the world's most valuable AI. It's TikTok's differentiation. It updates recommendations within 1 second of you clicking - at human perceivable latency. If your AI recommender has poor feature freshness, it will be perceived as slow, not intelligent - no matter how good the recommendations are.

TikTok's recommender is partly built on European Technology (Apache Flink for real-time feature computation), along with Kafka, and distributed model training infrastructure. The Monolith paper is misleading that the 'online training' is key. It is not. It is that your clicks are made available as features for predicitons in less than 1 second. You need a per-event stream processing architecture for this (like Flink - Feldera would be my modern choice as an incremental streaming engine).

* https://www.youtube.com/watch?v=skZ1HcF7AsM

* Monolith paper - https://arxiv.org/pdf/2209.07663

[eddd-ddde]

I have to say, it is _extremely_ impressive when a tiktok I watched reminds me of some other tiktok, so I go and search for a very loose description of the tiktok, and the first result is 95% of the time what I wanted to find.

I don't think any single other platform has as good a search feature as TikTok does.

[tridentboy]

oh wow, you're really lucky. around my friend groups who use tiktok, the main complaint is how bad the search is. unfortunately for us, getting a specific video is almost impossible =(

[nik_0_0]

Thats super interesting (I deleted Tiktok because it was too addicting!), but this is a common complaint about Instagram is that it feels impossible to find a reel based on keywords.

[dmix]

I noticed Youtube shorts also seems to update the feed based on how long the last video you watched. If you're scrolling quickly then stop to watch a dog video long enough the next one is likely to be another dog video.

[kgeist]

It creates a weird feedback loop: after I watch video A, it recommends a similar video B, and if I make the mistake of watching that too, it then recommends video C on the same topic. Suddenly my feed is nothing but Stranger Things shorts for two whole days (literally not a single video about anything else). Skipping or disliking didn't help, then somehow it went back to normal after two days.

[randysalami]

I’ve noticed the same thing and this creates such a negative user experience. Every short is a reaction test and if I fail, I get slop. Makes the whole experience very jarring (for better or for worse).

[datsci_est_2015]

For better or worse with regards to my addiction, my subscriptions are all either science channels or high effort / high production comedy skits (e.g. DropoutTV). I still get slop, but I never subscribe and it mostly remains background noise

[AreShoesFeet000]

That’s the point though. It may seem as if you’re not in control when scrolling, but you can adjust your behavior to get the content you’re looking for almost intuitively. That’s actually something good in my honest opinion.

[Jensson]

Why is it good that you need self control to not get slop? Its much better if you can just turn that off and relax rather than having to stay alert to avoid certain content that it tries to trick you to serve you more slop.

Distancing yourself from temptations is an effective and proven way to get rid of addictions, the programs constantly trying to get you to relapse is not a good feature. Like imagine a fridge that constantly puts in beer, that would be very bad for alcoholics and people would just say "just don't drink the beer?" even though this is a real problem with an easy fix.

[james_marks]

Basically, I want to set boundaries in a healthy frame of mind, and have that default respected when my self control is lower because I’m tired, depressed, bored, etc.

“The algorithm” of social media is the opposite.

[AreShoesFeet000]

It’s because content curation is inherently impossible to reach the same level of relevance as direct feedback from user behavior. You mix in all kinds of biases, commercial interests, ideology of the curator, etc, and you inevitably get irrelevant slop. The algorithm puts you in control a little bit more.

[Forgeties79]

I don’t agree tbh. This is part of how people wind up down extremist rabbit holes. If you’re just lazily scrolling it can easily trap you in its gravity well.

[AreShoesFeet000]

But you can get into extremist rabbit holes independently of control surface. Remember 4chan? Dangerous content is a matter of moderation regardless of interfacing.

[xp84]

I try to react as “violently” as possible to any slop and low-quality crap (e.g. stupid “life hacks” purposely bad to ragebait the comments). On YouTube it’s called “Don’t recommend this channel” and on Facebook it’s multiple taps but you can “Hide All From…” Basically, I don’t trust that thumbs down is sufficient. It is of course silly, since there are no doubt millions of bad channels and I probably can’t mute them all.

[MengerSponge]

They built a slop machine, not something tuned for positive UX.

https://en.wikipedia.org/wiki/The_purpose_of_a_system_is_wha...

[HappMacDonald]

At the risk of going off on a tangent about that maxim; I feel like it's just misusing the word "purpose".

Maybe it would be cleaner to state that a system has no purpose (at least not until it is sentient), instead it has behaviors. Then one can observe that the purpose of the designers or maintainers of a system simply happens to be at odds (or as AI safety researchers would say, are "out of alignment with") the behavior of the system.

That all of course presupposes that one can accurately deduce the purposes of the designers/maintainers.. In the case of TikTok, I'd bet that we are all in agreement that their purpose is nothing more nor less than maximal value-extraction from people wishing to express themselves with videos multiplied against an audience of people who wish to view videos multiplied again against advertisers who want to insert propaganda into eyeballs.

[direwolf20]

If a system is not fulfilling its purpose, the system is changed. If the system is not changed, it is fulfilling its purpose.

[pandemic_region]

I've been insta-skipping tennis video's for months now. Still getting Federer on a daily basis.

[beAbU]

Facebook does the same. The longer I dwell on an image post, the more likely the next batch of posts would be similar

[coliveira]

The right way to look at these networks is that people are being trained by the algorithm, not the other way around. The ultimate goal is to elicit behaviors in humans, normally to spend more time and spend more money in the platform, but also for other goals that may be designed by the owners of the network.

[touristtam]

Is amazon using the same thing??? I can't count the number of times I am getting recommended the EXACT same type of product I just purchased.

[beAbU]

On amazon.ie I'm convinced they are running only two ads, because all I'm ever seeing are ads for grime brushes and window squeegees. Literally nothing else.

[BoxOfRain]

One of my gripes with youtube at the moment is that they break my adblock filters to remove shorts more often than they break the filters stopping the actual ads.

[sidharthv]

I naively searched in the mobile app settings for a way to turn off shorts, before realising there will not be one.

[ffsm8]

You can't turn it off entirely, but if you keep using "show less shorts" from the 3 dot menu it eventually goes away, mostly...

[direwolf20]

It comes back. It acts like it executed shortiness+=1 every day, and "show fewer shorts" does shortiness-=10 or thereabouts. The shorts position on the home screen is based on this hidden shortiness variable. It always bubbles back to the top unless you keep pressing "show fewer shorts" whenever you see it.

[ffsm8]

Strange, for me it only comes back up if I get baited into clicking on any short (or have one linked to me)

[notpushkin]

You can use NewPipe or YouTube ReVanced, or set up a “child” account and disable Shorts in the parental restrictions settings.

[mghackerlady]

If you're on android it's better to just use revanced or something

[satiric]

I use an addon for firefox called "Hide shorts for Youtube™". Works just fine.

[exq]

I use this add-on to redirect shorts to the regular video URL. I can still watch them but I don't get the infinite scroll and algorithm pigeon-holing.

https://addons.mozilla.org/en-US/android/addon/youtube-short...

[rjh29]

youtube's algorithm seems to be "oh you watched this video? now here's every other video by this creator, pretty much without a break, until you downvote it"

It never reliably gives me videos similar but not exactly the same, i.e. things I might be interested in.

[sensanaty]

For me it's the same exact 5 videos on repeat, over and over and over again. I've gotten in a loop a lot of times, where it'll autoplay the same video I just watched, it's absolute madness

[vjerancrnjak]

Flink is too slow for this.

If by features you mean tracking state per user, that stuff can be tracked without Flink insanely fast with Redis as well.

If you re saying they dont have to load data to update the state, I dont see how massive these states are to require inmemory updates, and if so, you could just do inmemory updates without Flink.

Similarly, any consumer will have to deal with batches of users and pipelining.

Flink is just a bottleneck.

If they actually use Flink for this, its not the moat.

[btown]

Yea, the Monolith paper by Bytedance uses Flink but they only say it's in use for their B2B ecommerce optimization system. Maybe this is intentional ambiguity, but I'd believe that they wouldn't rely on something like Flink for their core TikTok infrastructure.

My hunch is we start to learn a lot more about the core internals as Oracle tries to market to B2B customers, as Oracle is wont to do!

[vjerancrnjak]

Flink is not really a performance choice, it's bloat to throw software as fast as possible at problems. I don't think there's any benchmark demonstrating insane capabilities per machine. I definitely couldn't get it to any numbers I liked, given other stream processing / state processing engines that exist (if compute and inmemory state management is the goal). Pretty sure any pathway that touches RocksDB slows everything down to 1-10k events per second, if not less.

The problem of finding out which video is next, by immediately taking into account the recent user context (and other user context) is completely unrelated to what Flink does -- exactly-once state consistency, distributed checkpoints, recovery, event-time semantics, large keyed state. I would even say you don't want a solution to any of the problems Flink solves, you want to avoid having these problems.

[lsuresh]

Thanks for the Feldera shoutout Jim.

For anyone else, if you want to try out Feldera and IVM for feature-engineering (it gives you perfect offline-online parity), you can start here: https://docs.feldera.com/use_cases/fraud_detection/

[bobek]

It is not only recommender though. These guys [1] seem to be able to react pretty quickly and not to create addicts on the way ;(

[1] https://recombee.com

[miohtama]

TikTok's differention is the userbase of all teenagers in the world.

[wongarsu]

But go just one layer deeper to 'why is every teenager using Tiktok' and the primary answer once again becomes 'Tiktok's recommendation engine'

[godshatter]

I'm not a TikTok user, but I'm assuming the recommendation engine is there to keep eyeballs on more ads for longer. Maybe we should be regulating how often and how many ads can be shown on social media, especially to teens and kids.

[cloverich]

They are arguing its not the recommender that is unique it is the network effect.

[wongarsu]

If it was only network effect, then how did TikTok grow in a space where Instagram and Youtube were already much bigger players? How did they gain that user base?

Network effect helps, but it only explains why they stay big, not how they got big

There really isn't that much making TikTok unique. Yes, their app is well designed. Yes, stitches and video replies make for great social/parasocial features because creators are actually interacting with each other and the community, almost like tumblr. But in my opinion those are reasons number three and two why TikTok is successful. Their recommendation algorithm is number one, by a wide margin.

[miohtama]

Because back in a day, Instagram and YouTube were shitty products and by some metrics still are

[expedition32]

No the primary answer is "teenagers do what other teenagers do". Remember we are advanced apes no more no less.

There is this curious word "influencer" which everyone uses but few ever think about what it really means.

[direwolf20]

That's not an answer, that's a thought–terminating cliché.

[AlienRobot]

It also provides different opportunities for growth compared to other social media. A video that gets over half a million views on TikTok may not get 5 thousand on Youtube, or even 10 views on Instagram or Facebook.

[bigfishrunning]

Isn't the inverse true though? it's not as if nobody's watching youtube, it's just that different videos are popular there.

[dehue]

It's not just different videos, Tiktok is much better at recommending videos by very small creators and people with no followers. On Instagram or Facebook if you don't already have a large following you most likely wont get any views at all no matter how well your video matches the platform. YouTube often pushes big creators that already made it big while Tiktok allows me to discover new and niche ones.

[notyourwork]

That didn’t by accident though.

[3abiton]

It's interesting to how they found out the "lifetime" of features is a feature by itself. Meta features is real.

[not_ai]

I’m happy to see that Flink is in this stack, I wish that Pulsar was as well instead of Kafka.

[permo-w]

I'm sorry to point out the obvious here, but who is going to perceive their recommended feed as slow or unfresh if it doesn't learn from exactly the last video you clicked on within 1 second? The bar simply is not that high. The special sauce of TikTok is how it chooses the videos, not the speed it does it at. I'm sure the speed helps to give it that "spookily intelligent" feeling, but that's a cherry on the recommendation cake, a cake which is already twice as good as the nearest competitor. I'm sure your talk goes deeper than this, but if this is the main focus, then you've missed the point.

[owenversteeg]

I partly agree and disagree.

Speed completely changes the game in a few ways. The first is identifying interests. Imagine every possible interest in a tree structure. Let's say you're into kumiko. There are so many levels of the tree to traverse to find kumiko; perhaps Skilled crafts -> Woodworking -> Japanese -> Construction without use of fasteners -> Panels and decorative elements -> Kumiko. The more iterations you can get through, the better you can match people's interests. If someone has 10 interests and each one requires many questions to determine, it can take forever to find exact interests with a system that only narrows down your interests every X videos vs. after each video.

The second is matching current moods. Let's say you just broke up with your girlfriend, or your pet fish died, or you're on vacation in Spain. A rapidly-updating system can capture those trends and get right to the heart of them in time for them to matter. A slow system might only get through a few iterations and capture a vague interest in Spain; a fast-updating one can get through countless iterations of guessing. Spain? What city? Tourist or moving there? What type of tourist? Foodie? What type of food? How fancy? Bam, you're watching the perfect video about an upscale seafood restaurant in Barcelona.

The third is type and flavor of content. Even inside of a small niche you will find many flavors of content. Super-short or long form, fast paced or slow, funny or serious, intellectual, irreverent, political leanings, background music, et cetera. Maybe you like slow long-form woodworking content but like fast-paced travel guides. Maybe you hate background music except when it's in skateboarding videos. To determine this requires an incredible amount of "questioning" of the user.

Now, of course, an algorithm that updates once daily can also make inferences about your interests and preferences. It can certainly learn, with enough time, what you are into and how you like to consume it. But the key thing is that these inferences only enable _predetermined_ changes. Imagine you are a human showing someone TikToks. Imagine that you can ask them any questions about their preferences right as they watch a video. You may not ask a question after every video, but you will ask countless questions over the hours of scrolling that day, and you will get good data. Now imagine a new restriction: you must decide your questions once a day in advance. You will manage far fewer questions; and to follow up on them you must wait yet another day.

Now, why do I partly agree? Well, I don't think speed is everything; I think TikTok has another sort of je ne sais quoi to it. I think it has a unique culture and community. It has a better UI and better features than Instagram. It has a young and cool reputation, far from the Millennial taint of Instagram or Facebook. And I suspect that they are good at identifying _who_ you are and acting on that information. But in my eyes, the speed could very well be the most important part of the puzzle.

[ryanjshaw]

Great insight. Any thoughts on RisingWave?

[jamesblonde]

That, too, and materialize. Feldera is my favourite, though.

[SpaceManNabs]

apache flink is so good. i think netflix used it heavily in 2018. not sure about now.

[cactusplant7374]

I thought was secret information. How long as it been publicly known?

[permo-w]

The secret is how it chooses videos to recommend, not the tech stack it uses and how fast it is

[cactusplant7374]

I would assume it's time spent on video and they start to build a profile of which users like what kind of content. X liked video 934934 so Y probably also likes that kind of video. Group people in buckets.

[permo-w]

I'm sure this is part of it, but I suspect it goes deeper than that. I'd guess they probably have some kind automated categorisation algorithm that can extract features from the videos