Hacker News new | ask | show | jobs
by slt2021 1246 days ago
the reason for that is near real-time detection of threats requires aggregation of terabytes of data according to rules (continuous GROUP BY on thousands columns on a sliding window) - and these aggregates by design have to be stored in RAM.

Otherwise these detections stop being near-realtime and become offline detection instead, just like any other sql server.

1 comments

To be clear - we were hosting on-premise, and being charged for our own RAM. Servers we had to buy, and then pay for the privilege of using with ELK.