| Honestly - I agree about the ELK stack side - piping all your logs into ES / Logstash is a great idea. (Or Splunk / Greylog / Logentries) If you run any sort of distributed system, this is vital. And while that counts as binary logs, I would argue that on the local boxes it should stay text. I would agree, if you are running any sort of complex queries on your data - go to logstash, and do it there - it much nicer than regexes. If on the other hand, you just want to see how a development environment is getting on, or to troubleshoot a known bad component tail'ing to | grep (or just tail'ing depending on the verbosity of your logs) is fine. I don't have to remember some weird incantation to see the local logs, worry about corruption etc. One problem I will point out with the setup described is syslon-ng can be blocking. If the user is disconnected from the central logstash, and their local one dies, as soon as the FIFO queue in syslog-ng fills, good luck writing to /dev/log , which means things like 'sudo' and 'login' have .... issues. Instead, if you have text files being written out, and something like beaver collecting them and sending them to logstash, you have the best of both worlds. |