Hacker News new | ask | show | jobs
by adrianN 2327 days ago
Just a few days ago I wrote a simple awk script to parse some log files but it was horrendously slow. I had to replace understandable loops with weird calls to builtin functions to make it fast enough for my usecase.
2 comments

You're doing something wrong. I've used awk to run big data reformatting jobs in under an hour that took most of a day to run in Scala on an Apache Spark cluster. In the vast majority of cases today, if speed is your problem, then you are the problem - especially since most problems fit into RAM these days, even w/o exotic stuff like RAMcloud...
try mawk, I've had it run 4x faster than gnu awk on some things.