[G_o_D]

why no mention of awk ? awk '!a[$0]++'

[noamteyssier]

I've added awk into the benchmarks also!

[pabs3]

Also perl, should be faster than awk IIRC:

perl -ne 'print if ! $a{$_}++'