Hacker News new | ask | show | jobs
by mattgreenrocks 4481 days ago
Preach it. My favorite persistence code: stuff that has nothing to do with SQL/NoSQL.

I leaned heavily on Python's pickle module for serializing a few thousand entities to disk a few years ago. By streaming them to the application at startup time, it remained plenty fast for all datasets it'd encounter. I intended to replace it with SQLite one day, but I never had to. I could just keep them all in memory.

I'd probably choose something a bit safer now, but it was hard to beat the simplicity.

1 comments

I used to do that, but pickle bit me once. I think it changes between versioning or something. I had to start the statistical model from scratch.
Now that you mention it, I remember running into something like it. It's a big issue. That module had a bunch of scar code to migrate entities as they came up from disk.
Yeah, I wouldn't use pickle for anything where backwards or forwards compatibility is important. It is however very convenient for 'I want to send/store this data for a bit' tasks.