[greatpostman]

I’ve built one of these at FAANG. Generally the different parts of the system are completely separate teams that interact through apis and ingest systems. Usually there’s a mix of online and offline calculations, where features are stored in a nosqldb and some simple model runs in a tomcat server at inference time, or the offline result is just retrieved. Almost everything is precomputed.

We had an api layer where another team runs inference on their model as new user data comes in, then streams it to our api which inboards the data.

On top of this, you have extensive A/B testing systems

[splonk]

I have as well, and your comment matches my experience more than the article does. Different teams own different systems, and there's basically no intersection between "things that require a ton of data/computation" and "things that must be computed online".

[oofbey]

Yep. The author, as a peddler of recommendations solutions, has an incentive to convince people that this problem is very complicated, and they should hire a consultant.

In practice, good old Matrix Factorization works really well. Can you beat it with a huge team and tons of GPU hours to train fancy neural nets? Probably. Can you set up a nightly MF job on a single big machine and serve results quickly? Sure can.

[jamesblonde]

I thought the two-tower embedding model was now the go-to approach for building real-time recommendation engines?

https://www.linkedin.com/pulse/personalized-recommendations-...

[random314]

In a certain sense, matrix factorization is a special case of a TTSN with RMSE loss.

If you want cross entropy loss, use word2vec.

Follow it up with a lambdarank ranker.

[nerdponx]

I think the question of "fancy technique versus simple technique" is beside the point. Assume for the sake of argument that you have a research organization and that it's worth your while to commit their time to a recommendation system.

The point here is that you don't typically need a huge amount of computation power to serve recommendations, even if the underlying model is sophisticated and required a lot of computation to train.

Likewise for data access, the online recommendation system typically does not need full access to the databases that the researchers need access to.

[lysecret]

Yea same here. What Nosql DB did you use for these lookups? Im currently using postgres for it but seems a bit like a waste. Even though the array field is nice for feature vectors.

[jenny91]

Presumably they mean internal stuff like google bigtable or equivalent. (Though some version of that is now on gcp).