Cloud Photo Storage

I recently sold my Synology NAS box, as I ran out of storage due to shooting with Sony A7r2 - all these raw files! I also moved to Amazon cloud, as it provides unlimited photo storage. I'd like to see some opinions or suggestions on where everyone is storing their photos?

For me, my primary source is Smugmug for jpegs, and amazon cloud for raw images. I was looking at other providers, but dropbox doesn't provide more than 1tb of storage. My issues with Amazon are the lack of solid app for Mac OS, as well as the web interface failing to display all of the galleries, and some other issues that keep appearing and disappearing. I'm looking for a solid and reliable way to store the images.

Looking for recommendations.

Why didn't you just expand the storage on the Synology? That's sort of the point of having a NAS.

Why would you trust ANY provider with your only copy of a file, then limit your access to Internet speed?

I keep everything on my NAS, backup to Crashplan, and use Smugmug for display/sharing.

mike_d wrote in post #18190400
Why didn't you just expand the storage on the Synology? That's sort of the point of having a NAS.

Why would you trust ANY provider with your only copy of a file, then limit your access to Internet speed?

I keep everything on my NAS, backup to Crashplan, and use Smugmug for display/sharing.

Why?!

Well, upgrading my outdated 214+ to something like 1515+ would cost me $700 for NAS enclosure, plus $160 per 4Tb drive. I played with an idea of building my own NAS, and actuall did it (FX-6700, 8Gb RAM, NVidia 740, 250Gb SSD). But essentially figured out this desktop was overpowered for pure NAS use, and using it as a desktop now. But the idea of building a FreeNAS kit has been in my mind for a while.

So aside from Synology that I've had before, any other options for storage / backup? In fact, I did have that exact combination of Synology + Crashplan, and later Synology + Amazon Cloud Drive (once the API started allowing multiple folder sync).

The delete key is really helpful to avoid filling your disks. I photograph weddings and my family, I delete 2/3 of what I shoot, and a good fraction of that gets converted to jpeg with the raw deleted. I don't need raw files for routine family stuff when I have a good jpeg.

I don't use a NAS. They use power, they have propriety file systems which are more difficult to recover from. I use internal disks, often in RAID, which I backup to CrashPlan. When they fill I archive to two offsite disks, and going forward I'll probably archive medium sized jpegs to AWS S3 / Glacier - I have a few weddings up there in that format and it costs me about 5c/month.

Never trust your data to one disk, one provider, or physical location. Every disk fails, even good providers can go out of business. When architecting large ICT solutions I routinely set up backups to another provider, just in case - on-site, Dropbox, Azure, Google, etc.

What happens to the Cloud when we have a massive Solar storm such as the Carrington Event of the 1850's? CD's or DVD's will still have the information on them but all magnetic media and most electronic gadgets will be out of service. Once rebuilt or new stuff made the CD and DVD images will still be available while just about everything not hardened/protected will be lost.

For those not familiar with the Carrington Event - a Solar Storm that was powerful enough to run electricity over telegraph lines and cause fires as a result.

When we get hit again we'll lose a lot of electronics. Looking to protect my images - the digital stuff. I know my film that is processed will be OK as it is stored in containers and buildings that are fireproof.

Cloud will likely fare the same as all electronic devices, though it will be much better protected than most, in purpose built facilities, and with professionals running things. If it only affects one part of the planet then you can spread your data out across multiple data centers to increase your chances. If what you say is true then tape, hard drive, and SSD could be lost, so there's bugger all anyone can do about it.

tim wrote in post #18202095
Cloud will likely fare the same as all electronic devices, though it will be much better protected than most, in purpose built facilities, and with professionals running things. If it only affects one part of the planet then you can spread your data out across multiple data centers to increase your chances. If what you say is true then tape, hard drive, and SSD could be lost, so there's bugger all anyone can do about it.

You could spend half your life burning and managing BD-R backups.

Oh, and there's always prints.

Deardorff wrote in post #18201901
What happens to the Cloud when we have a massive Solar storm such as the Carrington Event of the 1850's? CD's or DVD's will still have the information on them but all magnetic media and most electronic gadgets will be out of service. Once rebuilt or new stuff made the CD and DVD images will still be available while just about everything not hardened/protected will be lost.

For those not familiar with the Carrington Event - a Solar Storm that was powerful enough to run electricity over telegraph lines and cause fires as a result.

When we get hit again we'll lose a lot of electronics. Looking to protect my images - the digital stuff. I know my film that is processed will be OK as it is stored in containers and buildings that are fireproof.

I'll be quite happy with my backup hard drives stored in earthed metal filing cabinets. The results of both manmade EMP and also natural solar events are generally very over played by the media. You don't really even need to earth the box, and I would expect that most of the metallised bags sold for SSD protection should actually do the job as far as em protection goes. Personally I would expect the data on the platters to survive most events, since the cases should be pretty good Faraday cages in themselves. The control/interface systems may have problems as the connectors are outside of the cage, and so there is a very small chance they could be affected. The biggest issue is going to be the actual location of the disk being physically destroyed by fire, this is bad since very high levels of heat is much more likely to affect the magnetically stored data on the platter. As far as solar events are concerned unplugging devices from power and network distribution cables, and I guess actually removing device power leads where possible ought to be all that is necessary, It is the very long cable runs acting as aerials, allowing the build up of very high voltages that is the main danger.

During my nine years in the RAF as a radar tech in the 80s and 90s one of my tasks was both to oversee the emergency communications equipment, as well as deploying it. Before you become involved with this stuff you kind of think it is all very special stuff, the reality becomes much less impressive, it was simply a couple of HF radios stored in a standard metal office cupboard, with a precautionary earthing strap. Deploying it during exercises was usually just a bit of fun, well if the weather was OK, and most exercises were planned for the summer months, much better than being on guard duty in a sanger for hours at a time. Of course in reality what one really had to worry about was surviving the other initial effects of a 25+ kt bucket of instant sunshine happening near by.

Alan

tim wrote in post #18190774
The delete key is really helpful to avoid filling your disks. I photograph weddings and my family, I delete 2/3 of what I shoot, and a good fraction of that gets converted to jpeg with the raw deleted. I don't need raw files for routine family stuff when I have a good jpeg.

Do you delete once deliver Tim? For some reason, I'm very hesitant to delete the RAWs, even after a year or so. I did just go through my LR catalog and delete everything from this year that I didn't edit from my weddings and other sessions.

huntersdad wrote in post #18207583
Do you delete once deliver Tim? For some reason, I'm very hesitant to delete the RAWs, even after a year or so. I did just go through my LR catalog and delete everything from this year that I didn't edit from my weddings and other sessions.

I keep the keeper raws for weddings. I might shoot 2000 photos and keep 500. I still have a D700, 12MP, so they're not that big. I can get 120 weddings, RAW and jpeg, with albums, on a 2TB hard drive with room to spare.

The Cloud is a PITA.
I started with dropbox and gmail, canceled both, and using OneDrive now; paid $70 for 1TB+Office free for 1 year.

Ok, that's all fine, but... your upload speeds are limited and are 1/5 slower than download. In NYC, it's some 25/5mbps; I use Cat5e Ethernet cables, get some 30/5. Thus, if you have anything above 1gb folder it's much too long to upload. My uploads crashed or was too slow.
SO...
Here was my solution: I zipped(Winrar) all folders above 1gb into split files of 500mb each. So, now I upload 4x500mb when I am out or during night. Works fine so far. Ifcourse, you do end up with dozens of files on the HDD or in Cloud.
I no longer use USB thumb drives, but used SD cards as backups and placed them into bank safe box.
Will get me a 4TB desktop HDD and a 2TB portable soon.

Gomar wrote in post #18216787
The Cloud is a PITA.
I started with dropbox and gmail, canceled both, and using OneDrive now; paid $70 for 1TB+Office free for 1 year.

Ok, that's all fine, but... your upload speeds are limited and are 1/5 slower than download. In NYC, it's some 25/5mbps; I use Cat5e Ethernet cables, get some 30/5. Thus, if you have anything above 1gb folder it's much too long to upload. My uploads crashed or was too slow.
SO...
Here was my solution: I zipped(Winrar) all folders above 1gb into split files of 500mb each. So, now I upload 4x500mb when I am out or during night. Works fine so far. Ifcourse, you do end up with dozens of files on the HDD or in Cloud.
I no longer use USB thumb drives, but used SD cards as backups and placed them into bank safe box.
Will get me a 4TB desktop HDD and a 2TB portable soon.

You're talking about your connection, that's not a cloud problem exactly, but it is related.

RAR is a really old format, and by combining files you increase your risk, as one compressed archive corrupting takes out a lot more files. It also won't help much as images are typically highly compressed already. In my opinion (ICT professional with AWS cloud qualifications and decades of experience) this isn't the best approach. I suggest you just upload your files.

OneDrive seems like it's more for storage than archiving to me, but it's backed by Microsoft so it will be very reliable. Beware though, viruses or ransomware can encrypt or delete files that it can find easily, and OneDrive seems like a big target, since it's on every computer and you're logged in all the time.

Incremental backups are the best defense against viruses and ransomware. My backup workflow above links to the products I use. Because of the volumes of data most people create a hard drive is great for this. I also render my raw files to jpg and back them all up online, and I do back up some RAW files.

So I guess I'm suggesting your run a proper incremental backup program and store the backup files in OneDrive, as well as storing your files. It doesn't really matter if it takes a while to upload, so long as you have a good data allowance.

Tim - I thought you built your own FreeNAS system because of it's file system. Are you not doing that any more?

flowrider wrote in post #18225395
Tim - I thought you built your own FreeNAS system because of it's file system. Are you not doing that any more?

I considered building a machine for a NAS a couple of years back, but it ended up that to do it well it would need a Xeon processor, ECC RAM, etc, to run ZFS. It was just too expensive to do well. So I used my existing computer and upgraded to W10 beta which gave me similar capabilities with hardware I already owned.