[INTPenis]

We are so many that have been through this just like you.

My realization was also what others have mentioned, that I'm trying to use a search engine backend for log storage.

Specifically 3 months, which is the law here. That meant that ES had to keep 3 months of logs readily searchable. It's just not feasible when you're generating 25-30G logs each day.

I still use ES for search engines but I've stopped using ELK for logs unless it's for small environments.