[diggs]

If you're open to using something proprietary and commercially supported then depending on your scale you should check out http://storreduce.com. We're a growing startup in this space.

We do real-time compression, encryption (on the wire and at rest) and deduplication for object storage. We currently support an s3-compatible (including a full policy engine) api out the front end, and on the backend we can store to anything that exposes an s3 api (S3, Glacier etc.). Because of the s3-compatible interface we work with any existing client tools that work with s3.

We pride ourselves on our speed and scale. We can do 600mb/s sustained throughput and easily scale to multi-petabyte datasets. We typically see 95%-97% dedupe ratios on backup data. We support high availability clustering and replication (for example, replicate between regions for DR).

We don't currently support snapshotting but it's something we can implement relatively easily if people need it.

Our deployment model is based on a virtual appliance and can be deployed in the cloud or on premise. We can also do things like an on-premise writer (that only uploads unique, deduped data over the network), and a reader in the cloud to support cloud workloads or DR.

We have a real focus on backup to cloud in addition to supporting real time big-data use cases in the cloud.

Disclaimer: I work here - if you would like to contact me please feel free [e-mail].

[k3d3]

Sorry to say, but if the source is not available for a crypto tool, it's probably not worth looking at.

Best of luck though.

[Tepix]

Perhaps they can move the encryption into an extra open source module in the future.

[edwintorok]

Encryption should happen client-side [1], an S3-to-S3 gateway wouldn't help unless you deploy one on all client nodes. s3cmd and duplicity has support via GPG, but not all S3 clients will know what to do with those files.

[1] http://www.skylable.com/blog/2014/09/transparency-reports-se...

Disclaimer: I'm co-founder of Skylable

[diggs]

Our server software is typically deployed as close to the source data as possible. This lets us move only deduplicated data over the WAN. It also supports our encryption model where we encrypt using a pluggable key management service e.g. Amazon KMS or a on-premise HSM before any data leaves the customer site. This is essentially the same model as traditional tape and disk backup software within the datacentre.

Additionally we use HTTPS between clients and our server, and our server and the storage provider (e.g. S3), as well as being able to enable server-side encryption for S3.