Hacker News new | ask | show | jobs
by jamesblonde 3511 days ago
It's so bad that big companies, like Netflix, have to write a whole service to store a copy of the s3 metadata: http://techblog.netflix.com/2014/01/s3mper-consistency-in-cl...

Netflix have turned the S3 metadata eventually consistent problem into s3mpr metadata eventually consistent problem. The difference is that they can now inspect and reason about s3mpr's metadata.

Spotify have had to do the same thing for Google Cloud Engine. I can't help wonder if eventually consistent object stores will go the way of NoSQL databases, when a consistent, scalable hierarchical filesystem appears.
2 comments
Scaevolus 3511 days ago
GCS is a bit better, since individual objects are strongly consistent, but I've been bitten by the eventually consistent listing.
link
Yrlec 3511 days ago
Also on GCS, if you do a HEAD after DELETE on a bucket that is under lifecycle management it returns 200 instead of 404. Not really a consistency issue but it can really come and bite if you if you're not aware of it. GET returns 404 but HEAD returns 200.

I reported it as a bug but Google said it was by design. More specifically they said: "You are correct, if the versioning enabled in your bucket then the object metadata is saved as an archive object in the bucket [1].This is the reason you are getting 200 for your HEAD request."

link
kevan 3511 days ago
>I can't help wonder if eventually consistent object stores will go the way of NoSQL databases, when a consistent, scalable hierarchical filesystem appears.

Of course they will. Eventual consistency is a huge tradeoff that I don't think anyone would make if they weren't forced to.

link
yandie 3511 days ago
Then you trade availability with consistency. No system can't escape CAP
link
kevan 3511 days ago
Exactly. Availability and partition tolerance are often hard requirements, so we have to do all sorts of gymnastics to deal with eventual consistency. In personal projects where availability isn't a big concern and my most precious resource is my own time I tend to make different tradeoffs.
link
jamesblonde 3511 days ago
Maybe something like this - 1 million ops/sec on HDFS: http://www.logicalclocks.com/index.php/2016/10/14/hops-smash...
link