|
|
|
|
|
by chaxor
908 days ago
|
|
Are there any developments towards helping these efforts out by decentralizing / backing up parts of the archive and way back machine data?
I know torrents are hugely important in helping to decentralize and maintain many important files in academia and such (e.g. NN weights and CERN data) but I think cephfs is also trying to allow decentralized data storage with redundancy. It seems like there is some solution which can provide a huge data source to be decentralized over arbitrary number of nodes, where each node can hold or back up just some part of the data, and allow for a dashboard view that shows the level of redundancy over all of the data for each of its parts. |
|