[8n4vidtmkvmk]

If you ever figure it out, LMK. I don't think I've ever looked at logs more than about 24 hours old. Persistence and durability is not something I care about.

Errors, OTOH, I need a week or two of. But I consider these 2 different things. Logs are kind of a last resort when you really can't figure out what's going on in prod.

[CodesInChaos]

Here "log" means "append-only stream of small records". This isn't just about traditional logs (including http request logs and error logs). You could use it to store events for an event-sourced application, and even as the Write-Ahead-Log (WAL) for a database.

A distributed, but still consistent and durable log is a great building block for higher level abstractions.

[8n4vidtmkvmk]

That makes more sense. I suppose an audit log would also fit. I guess append-only backups wouldn't fit the "small" requirement though.

[CodesInChaos]

"Small" means 1MiB per record here. But a higher level abstraction could split one logical operation into multiple records. Just like FoundationDB has severe limits on its transaction size, while higher level databases built on top of it work around that limit.

The OP's blog post linked to this article, which explains some scenarios where this storage primitive would be helpful: https://engineering.linkedin.com/distributed-systems/log-wha...

This product offers two advantages over S3: 1) Appending a small amount of data is cheap 2) Writes are forced into a consistent order (so you don't need to implement Paxos or RAFT yourself). Neither of these are useful for backups. Raw S3 already works well for that usage-case, especially now that Amazon added support for pre-conditions.