[sepeth]

You should use highest protocol of pickle, here are the numbers in my machine:

    In [5]: %timeit pickle.dumps(a)
    1 loops, best of 3: 724 ms per loop

    In [6]: %timeit pickle.dumps(a, protocol=pickle.HIGHEST_PROTOCOL)
    100 loops, best of 3: 12.4 ms per loop

[omginternets]

Oh wow! This is more than I dared hope for, many thanks!

[darkxanthos]

What's the trade off here?

[AnkhMorporkian]

Backwards compatibility. Anything pickled with HIGHEST_PROTOCOL will be unreadable in anything below Python 2.3.

[Tenzer]

It might be worth noting that if you pickle data in Python 3 then it actually has protocol versions which aren't compatible with Python 2: https://docs.python.org/3/library/pickle.html#data-stream-fo....

[robzyb]

So my Ford Model T won't be able to read it?

(That is my humorous way of saying that Python 2.3 is rather old and therefore HIGHEST_PROTOCOL seems like it would be a desirable trade off for most.)

[StavrosK]

Yeah, what the hell? Even 2.5 is way too old, I generally only support 2.6+ nowadays. If something breaks compatibility with 2.5, no problem there. Hell, Python 3 has been out for years!

[AnkhMorporkian]

Hey, I'm with you. I use 3.4 on almost everything, and 2.7 at worst anywhere else. I weep for those stuck using Python 2.2.

[StavrosK]

No, I'm agreeing with you. I want to transition to 3.4 completely (mypy looks amazing), but there are some small Django libraries (third-party libs) that aren't yet compatible :(