Hacker News new | ask | show | jobs
by wiredfool 5410 days ago
Syslog* doesn't scale, at some point, but that point is at O(10) machines, not at 10 hits per second. Configuration is often key.

I'm on rsyslogd or syslog-ng, depending on the machine at this point. Configurations vary, but they both seem to hold up well to consistently writing 100 messages/sec (and peaks of 20k/minute when I get portscanned) on a VM without causing stuff to break.

The no fsync option is important, and really, fsync on important for kernel logs in the event of a crash, and not a whole lot else. Not mail, not messages, not syslog, and certainly not debug. You get (roughly) max 100 fsyncs a second unless you're spending extra money on your disks. That's a really damn small budget given that you mega bytes and giga cycles for other things.

The default configuration (at least in debian/ubuntu) writes entries in lots of logs. There's catchalls, there's mail.info, mail.err, mail.log, daemon, and such. You need to prune that down and make sure that you're not writing your debug logs in 2 or 3 places with fsync.

I'm finding munin and my log greppers are a whole lot more demanding on the box than syslog.