| I'll jump in here and share our current problem and how we are addressing it today. Hopefully it proves useful for whomever tackles this project! So we are an ecommerce company and we do our own photography for products -- and we do a lot of it. We average about 2-3 photoshoots per month, and each photoshoot can be upwards of 40GB before any processing. We shoot raw images and are starting to experiment more and more with video. After we do the post-processing work and start the video editing we can push 70GB a shoot easily. Our current workflow is like this: Shoot tethered to a laptop each day (typically each shoot lasts two days). At the end of the day, we upload the files up to our local NAS (Netgear ReadyNAS, just shy of 3TB) which is redundant. At the end of the shoot, we load a copy of the entire shoot on the local computers that will be editing so that they don't have to work directly from the NAS. Over the next few weeks while they edit the files, they will push the new files up to the NAS into specific folders. (This is where it gets hairy)
We also have an offsite NAS that is located in the business owner's house (about 6TB, but this one is not redundant). We used to attempt to upload files each night from the local NAS to an offsite NAS via rsync, but the pipe was just way too small. It would take days to upload a photoshoot, and everyone in the office suffered from slow internet while that happened. Now what we do is bring the offsite NAS onsite every few weeks and do a manual sync! Locally the sync only takes a few hours and then that night it goes back home with the business owner. Now since the local NAS is less than 3TB, we can't store all of our shoots on there forever. When we get close to full on the local NAS, we grab an external hard drive and archive old shoots (typically 2+ years old) to it, and then store that in our massive safe (on-site though). If we need access to just one or two files, we can retrieve them from the offsite NAS, but if we need access to quite a few of them, we can dig them out of the safe. So that's our workflow. Probably not bullet proof, but it has been serving us well for the last few years. If someone came up with a better solution, we would be all over it! |