Any advice for long term archival storage?

There is another option. I have been looking for longer term archival options for quite some time and have recently settled on paper. As my earlier post mentioned, there are truly very few photos that I want to last generations. It seemed to me that all technical options are transient, great for backups but needing maintenance every few years to stay current. So, I have started putting together books. I have only completed one (looking for a printer now) and have several others in the works, but paper (with care) can last for centuries without any maintenance and the only thing I have to worry about is that the English language may change making reading the descriptions difficult.

RDKirk wrote in post #19320723
I think you missed Wilt's point.

It's not a matter of whether in 30 years there will still exist among professional historians and data scientists the technology to recover data from optical drives, but rather that the technology won't still be available from Best Buy or Amazon in 30 years.

I can pretty much guarantee that when my descendants find the box with my stacks of optical media, they're not going to have a reader, and they're not going to check Best Buy or Amazon to find a reader. They're just going to toss the disks out.

I didn't, but I appreciate your point.

This thread is about archival storage. It's not about whether someone will care about any of our photos or data in 30 or more years. We can have a separate Nihilistic thread about that topic if everyone wants to. The question of this thread is regarding hardware and storage media for long term archival. Not about the value of the data being stored.

None of our photos matter ultimately. It's not about that.

But if your descendants did want to take a look at your archived media decades from now, they likely will be able to access it with common hardware like USB and optical media more likely than any other media and connection protocol in 30+ years.

Obviously this is going no where if the discussion is going to just become the a sad nihilistic future where no one cares to even look at your photos or data or whatever you want to focus on, instead of the technology itself since the thread is about that and not about your images not mattering to anyone (so why do it?). I'll return to lurking if there's no productive discussion to be had on this subject.

Very best,

I bought a crate of 500 Gb 2.5" SATA HDDs. It's the first thing people swap out when they get a new laptop. You can find them for nearly nothing. I put them in an enclosure and plug them in via USB.

For my rate of shooting, each one will hold 4-6 months of everything I shot, both culled and unculled. When I fill one up, it gets elaborately labeled and goes into safe storage, written once and hopefully never to be looked at again. This is in addition to live backup storage of about 2 years of unculled files + backups of everything I've ever culled and edited.

I don't know if it's the best option. Maybe I'll get burned in 20 years. Maybe one or more of the drives will degrade or otherwise fail in storage. Maybe new technology will make them unreadable. But it seems pretty safe to me. Frankly, I'm more worried about losing them in a fire (and I do protect them against that).

RDKirk wrote in post #19320723
I think you missed Wilt's point.

It's not a matter of whether in 30 years there will still exist among professional historians and data scientists the technology to recover data from optical drives, but rather that the technology won't still be available from Best Buy or Amazon in 30 years.

I can pretty much guarantee that when my descendants find the box with my stacks of optical media, they're not going to have a reader, and they're not going to check Best Buy or Amazon to find a reader. They're just going to toss the disks out.

^^^
And although it still might be available in stores today, already four households out of four (our daughters and their families) did NOT have a DVD reader in their multiple PCs per household, and none had never considered the purchase of an external DVD reader, even though it cost only $15!
Those are facts, not 'misinformation'. I am sorry you don't seem to want to accept the reality that occurs, Malveaux. It is historical fact, not merely my 'opinion'.

And media compatable readers isn't the only issue. We know that as O/S move onward, they leave behind data...

"old System 6/7 floppies were formatted as FAT16 or FAT32, OS X El Capitan will not mount them. "

Wilt wrote in post #19320752
^^^
And although it still might be available in stores today, already four households out of four (our daughters and their families) did NOT have a DVD reader in their multiple PCs per household, and had never considered the purchase of an external DVD reader even though it cost only $15!
Those are facts, not 'misinformation'.

I think the point is that optical hardware is very available and inexpensive and likely to be so for quite some time. Because some do not have it now, it is easily acquired. Now if you had chosen the CDC 9760 disk packs, I'm not sure the same could be said. I'm sure someone still has one but not too many do. More importantly, finding one is going to be a chore and if/when you do, costly. I'm still going with books for image archival. The original data is not there, but the image is.

MalVeauX wrote in post #19320738
I didn't, but I appreciate your point.

This thread is about archival storage. It's not about whether someone will care about any of our photos or data in 30 or more years. We can have a separate Nihilistic thread about that topic if everyone wants to. The question of this thread is regarding hardware and storage media for long term archival. Not about the value of the data being stored.

None of our photos matter ultimately. It's not about that.

But if your descendants did want to take a look at your archived media decades from now, they likely will be able to access it with common hardware like USB and optical media more likely than any other media and connection protocol in 30+ years.

Obviously this is going no where, I'll return to lurking if there's no productive discussion to be had on this subject.

Very best,

So you are saying that USB and optical drives will be around? Why not just stick with USB alone? What is the advantage of optical? (I know very little about computers)

duckster wrote in post #19320797
So you are saying that USB and optical drives will be around? Why not just stick with USB alone? What is the advantage of optical? (I know very little about computers)

Hi,

Optical media (that is at least archival level quality, not the cheapest possible stuff) has a longer shelf life and is more robust in general than other forms of storage at this time. Spinning hard disc platters fail over time the fastest of the physical media. Solid state drives (SSD) are more robust than spinning hard discs but they too fail over time. Volatile memory is also not a great way to store things for long periods of time, so flash memory for example is not a good solution for archival level storage. Note, archival level storage is what is being talked about in this thread; not just general storage. Archival is meant to last decades and not have a major vulnerability secondary to its medium. Optical media that is meant for archival storage uses more robust materials for the layers and means of writing the information (cheap optical media has organic compounds that can degrade over time). A good quality archival optical medium stored in a fairly safe environment will outlive everyone. This cannot be said of other common storage types (other than tape, but tape is not something an average person at home would be exploring). So I purposefully limited this to what is common to every day people off the shelf between the common options.

USB is merely the interface, it has no storage capability on its own, so its referenced as the means to interface the medium in question to a system that can do something with data. Of all the external interfaces over time and current trend moving forward, USB is the most universal, which isn't a cliche given its naming of "universal." If you compare previous interface connectivity for storage media for common consumers, they change over time. USB has changed a little, but it has retained backwards comparability and still is doing this even as we move forward a few generations. That's a big deal if you want a simple means to access information and connect mediums to a system to read/write data. So in the gamble of "future" means to access your archival stored data, USB is one of the better if not the best interface option right now going into several decades into the future, when you compare it to what else there is that a consumer has access to.

Neither require advanced knowledge of PC hardware or software to use. Burning CD's/DVD's/Blurays is not an uncommon cultural phrase, despite most PCs and laptops not having optical storage drives but they have external interfaces to add these via USB. It's the most simple way that is actually good enough for archival level storage of data with the best potential to keep hardware and access to that data for many years forward unlike other options. And I cannot stress this enough: its affordable and doesn't require a crazy learning curve.

The essential thing about true backup strategies is that it doesn't matter much if you have immortal data medium sets if you cannot access the data later on and this is the entire argument behind optical mediums and USB interface at this time. A backup is pointless if you cannot access the data in 30~40 years. In reality, most backup systems would migrate to new technology every 10 years or 20 years, but for the sake of argument, let's assume you never did that and this was "lost" and found by someone and needed to be accessed. Based on hardware history and interface history and current trends in hardware and interface connectivity moving forward, for common consumer level equipment, optical media and USB wears the mantle. And since its a simple inexpensive system, as technology changes over time, its so much easier and cheaper to continue to migrate data through this rather than buy into some more complex system with a higher cost and less likelihood of surviving as a living technology into the next few decades.

The data is out there, this is nothing new, this is old hat in that regard. The capacity you need largely dictates what your options are. For smaller data sets that are not enormous in file size, optical media handles that. If your needs are multiple terabytes per few months, then obviously something else is going to be needed. Most people on a forum like this simply don't need that kind of capacity nor the budget to get into something that can handle massive data sets, just keeping the information at a particular level for discussion.

But again, this is stressing archival long term storage. Not just every day storage or the idea of incremental backups that are replaced in a time period. Archival level implies very long term and the potential to sit for a long period of time and still be functional much later on and has a means to access it still. So this information is to be taken in that context. Not everyone needs archival level storage. It's just the topic of this particular thread.

Very best,

I still have some 3.5" floppy disks from 20 years ago. I don't have any idea what's on them and even if I cared, I'm not sure I'd go to the trouble of finding a floppy drive. I suppose those still available somewhere...

Steve

stevewf1 wrote in post #19320828
.
I still have some 3.5" floppy disks from 20 years ago. I don't have any idea what's on them and even if I cared, I'm not sure I'd go to the trouble of finding a floppy drive. I suppose those still available somewhere...

Steve
.

.
Yeah, Steve, it's easy to find any kind of old technology on eBay. . No one need fear that any old devices will ever become unavailable. . Due to the ease of selling online, all kinds of old batteries and gadgets and computer accessories will always be readily available.


.

Tom Reichner wrote in post #19320833
.
Yeah, Steve, it's easy to find any kind of old technology on eBay. . No one need fear that any old devices will ever become unavailable. . Due to the ease of selling online, all kinds of old batteries and gadgets and computer accessories will always be readily available.

.

It takes more than just access to the old technology, you also need all of the supporting infrastructure. You need the interfaces and controllers, drivers for them, software to decode the data stream. At some point the technology, even when hardware is available, becomes useless.

stevewf1 wrote in post #19320828
I still have some 3.5" floppy disks from 20 years ago. I don't have any idea what's on them and even if I cared, I'm not sure I'd go to the trouble of finding a floppy drive. I suppose those still available somewhere...

Steve

Unfortunately a 3.5" floppy, while it may work, will likely have degraded over time. They were based on magnetism and not meant for long term archival purposes (though some were made for that purpose, most consumers were not shopping for that, they shopped for cheapest price for use).

But, you can buy a USB interfaced controller for 3.5" floppy retail right now for a few dollars, brand new. This tech hasn't vanished, despite its age, because lots of industries still use them even with the advent of flash memory USB drives. So you can also still buy 3.5" floppies online. Crazy to think some old defunct technology is readily available on the shelf right now, and isn't defunct and is still operational.

The caveat? You may have to update its filesystem for your intended needs depending on what operating system you're using. But you can access old filesystems no problem. It doesn't just "go away." That's MAC/Windows mentality where you use what you're told you can use.

Tom Reichner wrote in post #19320833
Yeah, Steve, it's easy to find any kind of old technology on eBay. . No one need fear that any old devices will ever become unavailable. . Due to the ease of selling online, all kinds of old batteries and gadgets and computer accessories will always be readily available.

If you just hop on any technology retailing store and search, you'll find all these old drives with USB interfaces, brand new for sale. They're not ancient and aged dusty units, but brand new. Kind of interesting that despite everyone's dystopian future thoughts on tech, it's still there. Inexpensive too. All the common stuff we grew up with in the 90's when PC's really started hitting the common person's home is actually still available, new, and able to work today and you can still get medium today for it.

A lot of people in these discussions are only thinking from a very limited perspective and have no idea that industry and machines often use this old tech and are not replaced for many decades and so that tech is kept around. It doesn't just go away or get left behind as suggested here by others.

Your point about the devices being readily available online is absolutely correct. And to take it further, because we have incredible storage capacity, we don't have online resources for the software and information on this tech disappearing and instead its forever archived on multiple places around the world and you can search it up at sub-light-speed. Crazy.

gjl711 wrote in post #19320938
It takes more than just access to the old technology, you also need all of the supporting infrastructure. You need the interfaces and controllers, drivers for them, software to decode the data stream. At some point the technology, even when hardware is available, becomes useless.

I agree that at some point things do become useless. However, we're not there yet and not there for the foreseeable future yet either. Instead, the information and historical trends show that we're just doing better and better at this and instead of filesystems and tech disappearing faster as its replaced faster, it's instead adapted and spawns new market for more adapters to keep it relevant and accessible to common modern systems. You can buy a USB interface controller card with interface for nearly any media you want to access, new off the shelf, as a common consumer. And modern operating systems can read ancient file systems (the ones that think otherwise are just Mac/Windows users who are told what they can use).

The hardware and software needed to access old media is available right now on stuff that is 30 years old. But, most 30 year old media is not intact because the mediums they were stored on are not good for several decades of lifespan to keep bit perfect data. This is the bigger issue. Not the hardware and software, but the media itself! All the old magnetic based discs, tapes, etc, can be made to power on and read, but the data itself is not likely intact. So this is why archival media for long term has a focus on mediums that will last longer with intact data. We don't have much history showing that we lose access to information due to hardware and software trends (despite what people think this hardware stays around even 20~30 years later, up to current time); but we do have lots of historical data showing that our media that holds the data we are looking to access does go bad essentially. So our hardware and software is great and there's longevity, but the media that we store on is not!

Very best,

Storage is important but not difficult to achieve with a sensible approach.

As a hobbyist - Be discerning, recognise when you've taken something special or that you're really proud of and keep it. Delete the hundreds and thousands of other shots and you'll quickly limit the demands on your storage and archive systems.

As a pro - You'll only be keeping the good stuff and then only really need to do so until such time as it's delivered to or requires access from your client/s. Amongst that collection will be some golden stuff you may want to store separately but you'll probably have decided and starred it in Lightroom before sending to the client and again, the numbers aren't usually massive. Crème de la crème is worth hanging on to though of course!

In both cases (and aside from workflow storage which is a whole other issue)
SSD's are cheap and fast.
Online gallery systems are inexpensive and allows sharing and sales/printing facility.
Make a print or an album (why not, the oldest photos we all enjoy are in that format and it's well tested technology).
Finally, and perhaps controversially. Archive to Bluray - you can store a huge amount of stuff for next ot no money and hold them off site too if you are sensitive to panicky hording notions!

But again, ask yourself, is this worth storing? That's the key!

Wilt wrote in post #19320752
^^^
And although it still might be available in stores today, already four households out of four (our daughters and their families) did NOT have a DVD reader in their multiple PCs per household, and none had never considered the purchase of an external DVD reader, even though it cost only $15!
Those are facts, not 'misinformation'. I am sorry you don't seem to want to accept the reality that occurs, Malveaux. It is historical fact, not merely my 'opinion'.

And media compatable readers isn't the only issue. We know that as O/S move onward, they leave behind data...

"old System 6/7 floppies were formatted as FAT16 or FAT32, OS X El Capitan will not mount them. "

Four of four households! Well, I see no reason for a larger sample.

The answer to the OPs question is optical media. Whether or not any of the people in your 4 households will be capable of accessing the data on them is relevant to a few people.
Whether or not Apple O/S can mount FAT16 or FAT32 matters to about 15-20% of computer owners, assuming that all Apple owners don't also have a PC. Assuming some Apple owners also have a PC, reduce that percentage.

If optical media is not the answer, what is?
There has to be a best option as there are multiple options.

tmurphy said in the OP:

gjl711 said in the first response:

Asked...answered.

The OP was about archiving family photographs. That's what make's Wilt's response relevant, because the conditions of what the ordinary people comprising families will want to retain and how involved in technology they are willing to be are significant concerns.

Anybody know how to get data off a Hi-8 video tape cassette? I don't see a place to plug it in.