[withinboredom]

I mean, it would be faster to just import them into an in-memory sqlite database, run a `union all` query and then dump it to a csv...

That's still probably the wrong way to do it, but 10 minutes for a 20k line file? That seems like poor engineering in the most basic sense.

[isoprophlex]

It's a twenty line bash script. Pipe some shit into sqlite, done.

But the guy 'is known to get the job done' apparently.

[bee_rider]

Maybe he’s recognized something brilliant. Management doesn’t know that the program he wrote was just a reimplementation of the Unix “cut” and “paste” commands, so he might as well get rewarded for their ignorance.

And to be fair, if folks didn’t get paid for reinventing basic Unix utilities with extra steps, the economy would probably collapse.

[isoprophlex]

Clearly I'm the dumbass in this story, as we're all paid by the hour...

[bee_rider]

Clearly! He’s found a magic portal to the good old days when the fruit was all low hanging, and you keep showing up with a ladder.

[xnx]

csvkit and duckdb would also be good options. Any llm will spit out a one-liner for any type of join you can describe.

[strken]

I'd probably think of xsv, go to its github repo, remember it's unmaintained and got replaced by qsv, and then use qsv.