Hacker News new | ask | show | jobs
by marcinzm 2418 days ago
The systems I've seen basically break things into different services. Tied together with gRPC or Thrift which have code generators for most languages. So the Java backend simply makes RPC requests to a server running R.

Although in one case we had very tight latency requirements (ie: 10ms) so the ML results were pre-computed and loaded from a cache on the backend servers.
1 comments
imagiko 2418 days ago
I imagine I will be breaking down into different services as well. An ML "blackbox" that makes a call to the back-end for data and returns a result/prediction. This could happen through an API. What kind of API to choose is still open.

I'm not very sure what you mean when you say the ML results were pre-computed?

link
marcinzm 2418 days ago
If you're pulling data from somewhere automatically then you should make sure to define the data contracts well and that they're not changed. Also, A/B test things if possible. I've had issues in the past where the data pipeline view of the data and the API view of the data weren't the same. Or where a bug was fixed that resulted in the values of certain fields to change.

>I'm not very sure what you mean when you say the ML results were pre-computed?

We were scoring ads per page, and possible values for both were known ahead of time. So for each ad-page combination we generated the scores and then pushed them into a giant cache.

link