It's 2020, what is your backup solution?

MalVeauX wrote in post #19113606
That's interesting. I'm curious though, the tape is designed to last 15~30 years if housed in controlled environment. The capacity is certainly there with multi-Terabyte. So the medium itself is not going to be something to survive the environment in your home. Climate controlled vaults and the machines to read/write the tape vastly out-cost the medium of course. Tape has always been a major backup medium for archive, but it's not inexpensive and requires significant investment. Likely too significant for photography enthusiasts. But a very interesting point to look into that tape is still alive and well with massive capacity!

That said, I don't think I could foot the cost of the equipment needed to even use LTO tape and for off-site you'd need a climate controlled place to do it (hopefully just someone's house). If the medium itself requires specific temperature & humidity ranges to achieve 30 years, that means it would not be happy at all if exposed to the environment during a significant event. And the technical side of things, this would not be an easy tech for someone to just pick up from you, if you stroke out or keel over, and figure out how to get the data off of it.

Very best,

I believe the long term cost is price competitive if your TB count goes up. It’s the initial buy-in for the readers that is ultra expensive. It’s definitely a logistical thing for corporations or really tech savvy families or something. I imagine a cellar in some geographically safe part of the world would be the best way to do it, but that really depends on if you have family or infinite money to throw at the problem with owning property, maintaining it, and guarding it.

So yes, tape is primarily only as the third back up option to RAID, a networked or islanded repository slow speed in another part of your home, and cloud back up with a financially resilient company with great IT and Infosec backing (Google Drive?), or if you want to stick it to the man (big guv’ment and big data).

Also one more option than offsite backup is to get a fire proof cabinet (not sure if those are flood proof too though). And / or a bunker with one of those biochemical filters, solar and batteries, a genny, etc.

It just depends on how valuable you think your data is. ^_^

The biggest draw for LTO is the Airgapped, islanded nature to protect against ransomware, etc. If a company does frequent enough backups of their entire system, then when ransomware or a state actor threat vector breaches your security and wipes the floor with you, then you have a back up option that costs a lot less sometimes than paying the ransom (down time for the business can also cost $$$$$$$$$$$$$).

RDKirk wrote in post #19113618
Being in my late 60s, I've figured out that any of my images that might survive me (that is, those my survivors won't just throw away when I'm dead) are already on a wall, in a book, or in someone else's computer. Any images that reside only in my own data files simply isn't going to survive my passing. That hard truth modifies my choices of what I think should be in anything I'd call an "archive."

Which is to say: Practically nothing.

All the images I think are significant enough to last beyond me, I get distributed right away to the people who want to keep them beyond me.

Or at least I put them on a wall somewhere.

They don't exist just in a data file.

Yeah, that’s pretty much the USP of physical albums sold to customers. Data can be lost or corrupted, social and web platforms can come and go. Albums and prints are tangible and physical and can outlast one’s lifetime and be appreciated by future generations who may have different knowledge requirements.

RDKirk wrote in post #19113618
Being in my late 60s, I've figured out that any of my images that might survive me (that is, those my survivors won't just throw away when I'm dead) are already on a wall, in a book, or in someone else's computer. Any images that reside only in my own data files simply isn't going to survive my passing. That hard truth modifies my choices of what I think should be in anything I'd call an "archive."

Which is to say: Practically nothing.

All the images I think are significant enough to last beyond me, I get distributed right away to the people who want to keep them beyond me.

Or at least I put them on a wall somewhere.

They don't exist just in a data file.

This is a very valid point. The idea of having data (images in this context) survive us, only to be unwanted or not important to the receiver and the next generation. This is very true to an extent. I recently had a conversation about just this concept.

That said, I think there's another paradigm to this. Many of us were born into a world without photography at our fingertips, without internet sharing platforms, without the idea of carefree consumption of media. Most of us do not really have family photos beyond a handful, if that, of our families from just 4~5 decades ago, unless someone was an enthusiast back then. I lack most of my childhood photos, they're long gone on kodak film and a few polaroids. We had to scan most of any surviving photos we had from the 50's to the 90's basically. So many of us of course do not really worry about having photos laying around in the decades to come, of ourselves. But, this new generation is born into a world with internet, handheld cameras with instant sharing online, and consume more photos than we imagined. If you were to take your family's photograhpic history and put it on social media, hordes of people would thumb through it, even if it were not their own family. But in the context of family, our next generation may indeed have a much different idea of wishing they had images from back in the day of their family, carefree existing in a mega cloud and accessible over social media or something that we abhor or don't care about, but they do.

So the idea of our photos, especially personal family photos, surviving us and not being important to the next generation, I'm not sure if that's true, maybe from our paradigm from being born prior to this, but for everyone after this, hording thousands of photos and sharing them endlessly seems to be a thing now. So perhaps, more so than ever, it will be important to actually have photos of our loved ones from time periods before the advent of camera phones and social media.

Printing of course makes a physical media; but they're gone in a house fire, damaged in a storm with flooding perhaps. Also not really going to survive time (exceptions exist of course). This isn't an archival or backup solution unfortunately.

Very best,

idkdc wrote in post #19113620
I believe the long term cost is price competitive if your TB count goes up. It’s the initial buy-in for the readers that is ultra expensive. It’s definitely a logistical thing for corporations or really tech savvy families or something. I imagine a cellar in some geographically safe part of the world would be the best way to do it, but that really depends on if you have family or infinite money to throw at the problem with owning property, maintaining it, and guarding it.

So yes, tape is primarily only as the third back up option to RAID, a networked or islanded repository slow speed in another part of your home, and cloud back up with a financially resilient company with great IT and Infosec backing (Google Drive?), or if you want to stick it to the man (big guv’ment and big data).

Also one more option than offsite backup is to get a fire proof cabinet (not sure if those are flood proof too though). And / or a bunker with one of those biochemical filters, solar and batteries, a genny, etc.

It just depends on how valuable you think your data is. ^_^

The biggest draw for LTO is the Airgapped, islanded nature to protect against ransomware, etc. If a company does frequent enough backups of their entire system, then when ransomware or a state actor threat vector breaches your security and wipes the floor with you, then you have a back up option that costs a lot less sometimes than paying the ransom (down time for the business can also cost $$$$$$$$$$$$$).

Agreed, it's a good solution potentially in the scale of things that require this scale. I'm not sure it's an affordable solution for someone for their photography passion or business though, unless their business is enterprise level; or of course unless they're just a tech enthusiast. As you said, depends on how valuable the data is to that individual.

So I'm not so sure anyone around here will likely have experience with personal backups of their photography with LTO tape at least for now.

I'm really curious how people are doing it for their own photos (data) now, and what they're looking to do in the future. The affordable realistic stuff that the average joe can implement without having to really get involved with the nuts & bolts. Not everyone is into the tech with computing and storage, as much as they are simply enjoying photography after all. I'm trying to keep it in context of this forum of course, in terms of why I'm even bringing all this up, to put it into perspective again.

Then again, this is a real thing for people in media production with video & audio, they are producing hordes of data, multiple TB's worth, often. Photography makes the smallest dent in someone's storage needs likely, in this realm of media generation and storage. This is where the popularity of NAS and RAID level devices and mass storage comes back into things. It's quite often these days to find a vlog/video about some videographers and how tech savy they had to become to handle their massive mount of video content for both a working environment (and to not lose it all to a failure) and storage needs. And this is for some non-enterprise stuff. So while NAS/RAID solutions are common now, and even more popular to non-PC-tech-groups (like storage IT and security groups), they are mostly operating on uptime equipment and media; not backups (unless they are doing something to archive, truly backup their final data, not using just standard discs that will fail over time).

That said, RAID is not a backup (not at all), it's just redundancy (if it's anything other than striped and uses mirror or parity) that helps with uptime. The drives themselves fail over time and require replacement and the whole array can fail while rebuilding. It's a good standard for uptime with high capacity using inexpensive discs and inexpensive platform hardware. But it's not a backup. A NAS is just another access point, but the system being run isn't a backup either, even if it too is a RAID or similar redundancy based system using discs that will fail over time and require refreshing and rebuilding sequentially over time. They're great for uptime and fast access to keep things moving. They're however still not backup solutions.

It's an interesting concept actually. We all generate tons of data and don't want to lose it. But most people do not actually do real backups that will outlast them and be usable to someone else if they were to disappear from the equation. Maybe it's not important to us. But it likely is important to many others, since the tech is evolving, becoming affordable, and isn't a weird concept to most people these days.

Very best,

MalVeauX wrote in post #19113639
Agreed, it's a good solution potentially in the scale of things that require this scale. I'm not sure it's an affordable solution for someone for their photography passion or business though, unless their business is enterprise level; or of course unless they're just a tech enthusiast. As you said, depends on how valuable the data is to that individual.

So I'm not so sure anyone around here will likely have experience with personal backups of their photography with LTO tape at least for now.

I'm really curious how people are doing it for their own photos (data) now, and what they're looking to do in the future. The affordable realistic stuff that the average joe can implement without having to really get involved with the nuts & bolts. Not everyone is into the tech with computing and storage, as much as they are simply enjoying photography after all. I'm trying to keep it in context of this forum of course, in terms of why I'm even bringing all this up, to put it into perspective again.

Then again, this is a real thing for people in media production with video & audio, they are producing hordes of data, multiple TB's worth, often. Photography makes the smallest dent in someone's storage needs likely, in this realm of media generation and storage. This is where the popularity of NAS and RAID level devices and mass storage comes back into things. It's quite often these days to find a vlog/video about some videographers and how tech savy they had to become to handle their massive mount of video content for both a working environment (and to not lose it all to a failure) and storage needs. And this is for some non-enterprise stuff. So while NAS/RAID solutions are common now, and even more popular to non-PC-tech-groups (like storage IT and security groups), they are mostly operating on uptime equipment and media; not backups (unless they are doing something to archive, truly backup their final data, not using just standard discs that will fail over time).

That said, RAID is not a backup (not at all), it's just redundancy (if it's anything other than striped and uses mirror or parity) that helps with uptime. The drives themselves fail over time and require replacement and the whole array can fail while rebuilding. It's a good standard for uptime with high capacity using inexpensive discs and inexpensive platform hardware. But it's not a backup. A NAS is just another access point, but the system being run isn't a backup either, even if it too is a RAID or similar redundancy based system using discs that will fail over time and require refreshing and rebuilding sequentially over time. They're great for uptime and fast access to keep things moving. They're however still not backup solutions.

It's an interesting concept actually. We all generate tons of data and don't want to lose it. But most people do not actually do real backups that will outlast them and be usable to someone else if they were to disappear from the equation. Maybe it's not important to us. But it likely is important to many others, since the tech is evolving, becoming affordable, and isn't a weird concept to most people these days.

Very best,

Honestly, I think the only substantial cost an individual needs to worry about is buying quality hard drives mixed with some sort of cloud storage. I think both personally and from my perspective from an enterprise level, not enough money is spent on the hard drives themselves. People like to buy Western Digital Green for example instead of Gold, and that’s where failures happen. Simultaneous back up and thinking about your data is also super important, and it’s probably the worst thing one can do to buy into a proprietary RAID backup that depends on a central NAS to survive. For example, I’ve wasted money on QNAP to have errors writing, and corruption on the OS and hardware level affect the RAID 10 set up. I now prefer just hard drives without striping nowadays, and NVMe drives for anything that needs super high speeds. A lot more peace of mind and not dependent on any particular operating system instance.

The argument I have against optical media is twofold...

• It is subject to the same organic dye deterioration from chemiical fumes, etc. as we have to protect our color film collections from
  
• I have actually experienced TWO commercially produced optical disks containing software (that came included with devices like scanners, etc.) that stopped being installable some years after I originally received and installed them...an early version of Photoshop LE from Adobe comes to mind.
  

Wilt wrote in post #19113645
The argument I have against optical media is twofold...

• It is subject to the same organic dye deterioration from chemiical fumes, etc. as we have to protect our color film collections from
  
• I have actually experienced TWO commercially produced optical disks containing software (that came included with devices like scanners, etc.) that stopped being installable some years after I originally received and installed them...an early version of Photoshop LE from Adobe comes to mind.
  

Optical disks just aren’t that storage-dense either. And yes, I agree, software will discontinue support pretty quickly thanks to Apple and companies iterating technology over time. Non-profitable and heavy, bloated, slow-moving companies like Roxio will die if they haven’t already.

Wilt wrote in post #19113645
The argument I have against optical media is twofold...

• It is subject to the same organic dye deterioration from chemiical fumes, etc. as we have to protect our color film collections from
  

Look into M-disc. That's not how it works. It's engraved.

That said, even a basic DVD-R or BD-R disc will outlast on average a typical hard drive or flash thumb drive over 25 years, with the dyes you mentioned.

What do you backup to Wilt?

Everyone has their experience and needs. I'm curious what everyone is doing, if they're doing anything.

Very best,

idkdc wrote in post #19113643
Honestly, I think the only substantial cost an individual needs to worry about is buying quality hard drives mixed with some sort of cloud storage. I think both personally and from my perspective from an enterprise level, not enough money is spent on the hard drives themselves. People like to buy Western Digital Green for example instead of Gold, and that’s where failures happen. Simultaneous back up and thinking about your data is also super important, and it’s probably the worst thing one can do to buy into a proprietary RAID backup that depends on a central NAS to survive. For example, I’ve wasted money on QNAP to have errors writing, and corruption on the OS and hardware level affect the RAID 10 set up. I now prefer just hard drives without striping nowadays, and NVMe drives for anything that needs super high speeds. A lot more peace of mind and not dependent on any particular operating system instance.

Agreed; I use enterprise HDD's, they're affordable. And now enterprise SSD is becoming affordable and the capacity is finally getting realistic for media needs (photography, videography).

In reality, for an uptime system, any basic setup with redundancy and data validation will do a good job of maintaining uptime for someone's data. Even though its not a backup that will last 20~30 years, or more. Most people will surely be completely covered for their actual needs with a living system (they work on), a redundant system using quality discs (that are refreshed periodically) and some minor cloud usage as you pointed out. I'm curious if they are concerned with actual backups though, in case things go wrong, such as controller failtures, bad RAM, etc.

Very best,

MalVeauX wrote in post #19113653
Look into M-disc. That's not how it works. It's engraved.

That said, even a basic DVD-R or BD-R disc will outlast on average a typical hard drive or flash thumb drive over 25 years, with the dyes you mentioned.

What do you backup to Wilt? Do you backup?

Very best,

Optical is super stable. It’s just the reader and software support side that is risky, in addition to lower density of storage that is no longer cost-competitive as you scale up.

For RAW or JPEG selects, this is as good as it gets though without cloud or LTO.

idkdc wrote in post #19113656
Optical is super stable. It’s just the reader and software support side that is risky, in addition to lower density of storage that is no longer cost-competitive as you scale up.

For RAW or JPEG selects, this is as good as it gets though without cloud or LTO.

This is true, the cost for capacity is significant so it's not a great option for someone looking to do backups of 40TB of data. A pool of data not uncommon to someone doing videography often such as high quality production video with audio. The working data itself is massive. And archiving it would be very expensive. Most of these of course opt to use inexpensive disc based arrays with redundancy.

I'm curious though, the reader and software support being a risk with optical, but is it though? Think about it. Every bit of optical since the 90's is commonly read in common hardware today without any special hardware (thanks to ISO and UDF formatting on the discs, not operating system or file system dependent). So far, we have 30 years of track record of optical being available and easily accessed with basic hardware that is still produced and available and not even unique to a platform (ie, works fine for Mac, Windows, Linux, etc). Mean while, hard drives, and other formats like tape, have changed their connection types and interfaces and are far more risky, because you'd have to find that hardware to convert it for today's hardware. While it is still available, you can totally still find a way to hook up an old IDE drive or SCSI drive 30 years later, that's a lot of fuss, where as a CD-R or DVD-R from the 90's needs nothing; you can literally go pick one off the shelf and put it in your modern computer's optical drive and it will work (unless its degraded of course) without any special software. This is pretty fascinating actually.

I'm curious which formats will be able to do this in the next 20~30 years.

Very best,

I have triple redundancy (actually, 5 drives have the same data!). Any data worth keeping long term which is on the PC harddrive gets

1. copied onto a RAID 1 configured NAS (Synology) connected via ethernet and using enterprize drives
   
2. copied onto a non-RAID NAS connected via ethernet
   
3. copied onto a USB 3 connected RAID 1 (Buffalo)
   

MalVeauX wrote in post #19113662
This is true, the cost for capacity is significant so it's not a great option for someone looking to do backups of 40TB of data. A pool of data not uncommon to someone doing videography often such as high quality production video with audio. The working data itself is massive. And archiving it would be very expensive. Most of these of course opt to use inexpensive disc based arrays with redundancy.

I'm curious though, the reader and software support being a risk with optical, but is it though? Think about it. Every bit of optical since the 90's is commonly read in common hardware today without any special hardware (thanks to ISO and UDF formatting on the discs, not operating system or file system dependent). So far, we have 30 years of track record of optical being available and easily accessed with basic hardware that is still produced and available and not even unique to a platform (ie, works fine for Mac, Windows, Linux, etc). Mean while, hard drives, and other formats like tape, have changed their connection types and interfaces and are far more risky, because you'd have to find that hardware to convert it for today's hardware. While it is still available, you can totally still find a way to hook up an old IDE drive or SCSI drive 30 years later, that's a lot of fuss, where as a CD-R or DVD-R from the 90's needs nothing; you can literally go pick one off the shelf and put it in your modern computer's optical drive and it will work (unless its degraded of course) without any special software. This is pretty fascinating actually.

I'm curious which formats will be able to do this in the next 20~30 years.

Very best,

I think with all mediums, including LTO, it’s important to also have back ups of the the reader, the computer hardware and specific OS version and island it (to prevent hacking from an out of date service) so that you have previous version of OS’es available in case there are updates that phase out older equipment. This is not as much of an issue with standard computers that take disk drives, like Windows and Linux builds that take standard disk readers. This is not an issue within one’s lifetime with less important data, probably, too. It’s more of an issue for national/cultural, corporate back up, or perhaps even backing up of something of immense importance to you such as a digital journal.

I think with Apple’s push to drop standard Blu-Ray support on most consumer computer models (Apple has been the standard setter for years including USB from PS/2), this is a risk that hasn’t yet been proven, but to me it still seems like a non-zero risk. There are also issues if you pick the wrong standard, for example, UMD or HD-DVD, but I think those are all issues you can alleviate by transferring the data if you’re actively maintaining those optical back ups.

Optical disks should be low cost enough at low-density to have identical booklets too in two locations of a house.

From the software side, I think my main annoyance is with specific standards other than simple FAT-32 storage (again, an issue with video files than photos files which can be under 4 GB). For example, ExFat formatted storage drives created on Windowsnot (in my anecdotal experience) being compatible with Linux and Mac, etc. Little bugs in future updates that are esoteric with very few customers reporting can wreak your back up retrieval plans. In addition, using software from old days or technology that dies or is discontinued such as DVD Menu support by Adobe Encore DVD, is kind of wonky. Not so much an issue for data, but in case you count on a specific DVD or Blu Ray player standard to support a certain UI presentation. There is no telling what future players are compatible with.

Wilt wrote in post #19113666
I have triple redundancy (actually, 5 drives have the same data!). Any data which is on the PC harddrive gets

1. copied onto a RAID 1 configured NAS (Synology) connected via ethernet
   
2. copied onto a non-RAID NAS connected via ethernet
   
3. copied onto a USB 3 connected RAID 1 NAS (Buffalo)
   

So my LR files and RAW files are in 5 places.

And recently I upgraded the harddrives in the Synology, and the original harddrives were put into storage.
Admittedly I have not taken the archival measures of using cloud storage services (assuming the service remains in business...although the industry is quite spotting in that regard!

My assumption is that being 140' above sea level, and in the hills above my community, the risk of flood is low. So I need to worry only about earthquake and fire. Since I am on a rocky hill, it isn't likely that the ground will split and swallow my home. That leaves fire as the sole risk. Not being a worry wart, I don't worry about that.

And as pointed out, the likelihood of a relative surviving me and wanting my photos (other than the ones of family members), is somewhat in doubt. So that leaves historians or anthropologists who might have interest in seeing photos of history and lifestyle in the early 2000s.

You have one threat vector that you’re not prepared for, and that is malware or cyberattack. Large corporations have a ton more IT staff to counter this (although not perfect, they have more resources). An individual has obscurity on their side, but security through obscurity is a game of chance and taking risk. An optical solution or a resting hard drive that is air gapped (non-networked) is a really good way to deal with this without resorting to cloud.

Also, I wouldn’t recommend backing up to cloud all the time depending on the cloud service’s security level if you deal with extremely sensitive data like nudes, NDA materials which would have you in breach of contract if the data got leaked, or if you’re a real-life Heisenburg character. Think the iCloud celebrity hacking conundrum.

MalVeauX wrote in post #19113653
Look into M-disc. That's not how it works. It's engraved.

Interesting, that PCs (particulary the laptops that are so common (unlike the less common desktop PC in homes) today so SELDOM come with optical drives in them. 3 years ago, I gave all of our daughters families a USB connected DVD read/write unit, because NONE OF THEM could read the optical media with photos that I distributed to them after a 'family' vacation gathering somewhere.