Hacker News new | ask | show | jobs
by HappMacDonald 350 days ago
One day I wanted to use a TAP parser for the Test Anything Protocol.

But I didn't want to be bogged down by dependencies.. so I didn't want to go nowhere near Python (and pyenv.. and anaconda.. and then probably having to dockerize that for some reason too..) nor nodeJS nor any of that.

Found a bash shell script to parse TAP written by ESR of all people. That sounds fine, I thought. Most everywhere has bash, and there are no other dependencies.

But it was slow. I mean.. painfully, ridiculously slow. Parsing like 400 lines of TAP took almost a minute.

That's when I did some digging and learned about what awk really is. A scripting language that's baked into the POSIX suite. I had heard vaguely about it beforehand, but never realized it had more power than it's sed/ed/cut/etc brethren in the suite.

So I coded up a TAP parser in awk and it went swimmingly, matched the ESR parser's feature set, and ran millions of lines in less than a second. Score! :D

1 comments

For the record, "python-without-extra-dependencies" is a thing and a very nice one too. I always prefer it over awk.

Highly recommend to everyone - plenty of "batteries" included, like json parser, basic http client and even XML parser, and no venv/conda required. Very good forward compatibility. Fast (compared to bash).

That does sound compelling, but I frequently enough have to work on embedded systems (often running busybox) where python isn't installed and available R/W storage space is measured in tens or hundreds of Kb.

I find that that is one more environment where awk scripting can get the job done, python/perl/php/etc just can't be introduced, bash can _sometimes_ get the job done if it doesn't have to spawn too many subprocesses, and C/other-compiled-options _might_ be able to help if I had some kind of build environment targeting the platform(s) in question and enough patience.

I'll keep an eye out for python with no extra dependency options on the platforms that can handle that though.

Lua is pretty popular in the embedded devices I've been working on.. of course it's stdlib is tiny compared to python's though.
Can you explain more or provide more information/links?