Hacker News new | ask | show | jobs
by iradik 5387 days ago
haha.. i've gotten those looks as well.

but i agree. files and folders are an elegant abstraction, that when combined with the unix toolset become extremely powerful.

The big shortcoming I see with this solution, and maybe this is what you are saying in the caveats, is that it doesn't support multiple worker boxes.

Of course you could use NFS, but this complicates it. Suddenly the consistency model is more complex and workers must partition work, and so on.. At that point, a mysql backed queue becomes an appealing and easy way to make a distributed queue.

1 comments

My experience is that when something is filesystem based, you eventually have someone write a not-robust-enough bash script to do some maintenance operation (find|xargs|rm cleanup script, a sed based update script, etc) and it blows stuff up.

I think the transaction log and the forced structure of using SQL (barring some yutz carelessly using TRUNCATE) add some value managing the data, too. Not as big an issue where it's a single person maintaining the app.

i agree with your point. yet i've seen people make the same mistakes with sql too (they have autocommit=on haha).

hopefully whatever solution you have is tested and designed defensively so you don't accidentally rm the queue.