Hacker News new | ask | show | jobs
by Nukesor 822 days ago
Interesting.

May I ask how many tasks you were managing with Pueue and what your usecase was?

I also thought about using alternative formats such as CBOR, but choosing a human-readable format like JSON made debugging and such a lot easier.

If there's a good usecase for it, I might consider switching to a more compact format.

1 comments

I was running a lot of small tasks. With 3000 queued tasks my state file was around 25 MB.

zstd compressed that down to 5% of the size. I have the code still if you want to look at it but it was just a quick experiment so I didn't add any tests. I did add it to the config, disabled by default, though.

https://github.com/veyh/pueue/commit/e9dcf52227304b4b4a2ded4...

Protobuf could be a pretty good alternative. It can be dumped to a human-readable format with the protoc cli.