Hacker News new | ask | show | jobs
by TheDong 3769 days ago
Let's say that you're an irc network operator trying to make the channel's entire history searchable (e.g. botbot.me type thing).

For absolute correctness, you'd want to record per-leaf and merge.

However, the last time I needed log-merging functionality was much more boring; some of my log-files were corrupted, and someone else had logfiles that weren't as extensive, thus I needed to munge the two together. The timings were subtly different (because that's how irc works), so I wrote custom code to munge them together, preferring his for the range of corruption.

So yes, I've needed that functionality, though it wasn't actually done due to a netsplit. I can foresee it being useful in the case of a netsplit if your logging is at the ircd level or you run one bot per leaf.

1 comments

I wonder whether any large network keeps logs of channels, let alone one per server. And if they do, they probably don't have much use for a merged log except for taking up less space.

My own IRC log tool [GH: tilpner/ilc] can be used to merge logs, but I rarely use that functionality.

For two log files "a" and "b", in weechats default log format:

    ilc sort -f weechat -i <(cat a b) | ilc dedup -f weechat
I've never tested this with logs over 200MB, but sort will read the combined log into memory, which is definitely not optimal.