[bartonfink]

"Write specialized string parsers in C for doing simple things you'd normally do with a script. If you do it right, it'll be several orders of magnitude faster."

Note that this ONLY makes sense in the context of a learning exercise. If it's simple and you'd normally do it with a script, the performance you'd gain from a C rewrite isn't worthwhile. The knowledge you'll gain from making C do something it doesn't make easy, though, is.

[rbranson]

In most situations, I definitely concur. There are a few cases where I've had to process >10g flat files and I whipped up a specialized tool in C that could run through it nearly as fast as the disk could feed it.

[bartonfink]

Sure, but I doubt you'd seriously try to handle that with pipes and Unix anyway. 10g warrants someone who knows what they're doing and knows, like you, when the overhead of writing in C pays for itself in faster processing.

[ansgri]

Unless you're in scientific community, where it's common to have to convert GBs of data from one weird format to another, or possibly analyzing them, nevertheless knowing only e.g. R and Matlab.