[skrebbel]

Did anyone yet use Redis streams to store actual logs? Like server logs, application logs, etc.

I understand that Elasticsearch is a common place to put logs, also because I assume that searching through logs is a common use case, but I wonder whether Redis has particular benefits for this use case. The data structure seems particularly tailored to it (but not so much to searching I guess).

[antoncohen]

Log volume can easily exceed reasonable memory sizes. Even a small company can generate TBs of logs each month. Having a single box with TBs of memory wouldn't be desirable.

For logs without full indexing, Loki (https://github.com/grafana/loki) is a recent entry into the space, and it probably a good option to look at. It indexes metadata (labels), so it allows searching by labels but not full text. It is also supposed to be horizontally-scalable, which is probably something you want in a log storage solution.

[atombender]

My guess is that this would work fine until the working set size exceeds available memory. Redis (unless something new has happened the last couple of years since I used it) requires that data fit in RAM. So could work well for low-frequency logging like alerts. Not as a general purpose log system.