Hacker News new | ask | show | jobs
by richardknop 3109 days ago
Obligatory plug of my own job queue in Go: https://github.com/RichardKnop/machinery
1 comments

Faktory is CouchDB, Machinery is Redis/Memcache/MongoDB, Sandglass is a custom Raft on top of RocksDB…

I'd love to see comparative stress tests, Jepsen-like, to assess the ability to survive partitions, corruption, node restart and node loss, to better estimate the probability that a job gets lost.

Machinery is basically a Go implementation of celery (popular Python task queue).

It has two core components: broker (AMQP or Redis supported) and backend (Memcache, Redis or MongoDB or even no backend if you don't care about storing task states).

I think comparison with Sandglass would not be valid as Machinery is a higher level job/task queue while Sandglass is a lower level library (basically a message queue such as RabbitMQ which would be just a component of Machinery).

Faktory vs Machinery could be compared as they are on more or less same level of abstraction.

Faktory is based on rocksdb actually.