Storage Options

I’ve been searching for data/storage options and trying to understand all the technical information (overload) in my attempts to make a decision on what direction to go in. My situation is this: I’ve got two internal 5TB drives and have been manually (though automated) copying my photo’s every week. I set this up years ago and it’s worked reliably, with my biggest fear being either operator or mechanical failure. I’m now at the point where I’m out of disk space and reconsidering my options once again. What I’m looking for is a relatively cheap solution to backing up my photo’s with easy access to recovery should I need it. I haven’t been concerned about viruses, though after reading through the plethora of posts on the subject maybe I should be. I was advised at my local Microcenter to install a network Raid system consisting of 4 drives, which would act as a better backup than the system I’m currently using. This would cost me a little over $500, and adds the benefit of being accessed anywhere in my house. Conversely, I’m reading about cloud systems such as Backblaze and feel that maybe this is a good time to consider switching to this type of a system. I could backup all of my photo’s to the cloud and then convert my duplicated hard drive to primary storage, effectively doubling my capacity with no major outlay, though I would begin to incur a small monthly cost I don’t currently have. Anyone see any problems going this route? I’m a firm believer in learning by others experience (mistakes).

There are many different options and no option is best for everyone. However, if you have filled a 5tb drive, your first option might be to do a thorough housecleaning. Go through old libraries and delete files that will most likely never be used. I have recently completed that as I was in a similar situation, though with only about 2.5tb of images. After a really good cleanup, it's less than a 1tb and that includes a bunch of scanned tiff files each being about 100mb per image. I choose to go the hard drive route. I do have a raid setup internal to the PC as a first line of defense, and a few external hard drives I copy things to.

I personally don't trust 'cloud' storage options to remain available indefinitely,
and as you've already mentioned, it incurs monthly/annual costs not previously spent.
There's plenty of history of cloud providers suddenly shutting down.

I've had a fairly recent HDD crash that's got me looking into NAS and similar.
I want it to be in my possession, right here at home, not on someone else's machine.
I was looking into NAS offerings to begin with, but you're looking at the cost
of a full current PC system just to add drives to an empty NAS box.

I'm leaning towards building my own JBOD(Just a Bunch Of Disks)
with something like FreeNAS running it. It's more than likely I can cobble together something
reasonably effective to begin with by using a retired PC and adding drives to it, with such a system
being easily and inexpensively upgraded as needed in the future.
My thinking is to grab an inexpensive rack-mount chassis, throw MB-RAM-CPU into it,
and get to work. The chassis may have room for 4 drives, or as many as 10.
It's up to me to decide capacity and all hardware.

The beauty of a JBOD box to me is that they're almost infinitely configurable.
Drives, arrays, drive quantities, etc... All can be configured as you wish.
RAID-whatever or not, number of drives, memory, processor, etc.
is all up to you, not some manufacturer of a dedicated device.
The FreeNAS JBODs can even be chained and mirrored, as far as my understanding goes,
without any of the hardware, except drive capacities, needing to be matched,
and only if a RAID or other mirrored setup.

There is no failsafe backup plan, all you can do is try to backup your files in at least two places. Myself, I have paired external drives which mirror each other that I copy files onto after every project plus all studio paperwork and another that I run Time Machine exclusively. I chose the GTechnology Mini drives because of the reviews and are easy to stack on my desk and store in a lockbox. In nearly twenty years, only one drive has failed and it was easy to simply replace and copy from the matching drive. The upfront investment is higher on a NAS system and a two bay unit may be a good start. A cloud system depends greatly on your upload/download speeds and you will also need an onsite backup for insurance. What is your strategy if you have a burglary, fire or natural disaster?

SkedAddled wrote in post #18959764
I personally don't trust 'cloud' storage options to remain available indefinitely,
and as you've already mentioned, it incurs monthly/annual costs not previously spent.
There's plenty of history of cloud providers suddenly shutting down.

I've had a fairly recent HDD crash that's got me looking into NAS and similar.
I want it to be in my possession, right here at home, not on someone else's machine.
I was looking into NAS offerings to begin with, but you're looking at the cost
of a full current PC system just to add drives to an empty NAS box.

Yes, reliance on cloud service providers is murky at best. Even the 'big boys' get in and get out again from offering the service.
From a list compiled in March 2019...
https://photography-on-the.net …showthread.php?​p=18836089

Plus, getting your data to/from the cloud is inherently bottlenecked

"Backing up locally can literally be 1000 times faster – and local restore is about 300 times faster. Important statistics to know when a server goes down. Let’s take an example. How long do you guess it would take to upload 100 Gigabytes of data if you had a fairly typical DSL with 3Mbps download/512Kbps upload speed? According to the calculator (Refer to the bandwidth calculator link at the very bottom of this post) the FASTEST you could ever experience would be 19.5 days! In reality you should assume 50 to 70% of this number (at least a 30% to 50% reduction) for real world connection overhead. "
https://www.high-rely.com …oblems-with-cloud-backup/

As for the comment in blue, there are products such as from Synology which are NAS RAID products which required NO SEPARATE CPU. Just plug it into your router. I have had one now for several years.

Wilt wrote in post #18959810
As for the comment in blue, there are products such as from Synology which are NAS RAID products which required NO SEPARATE CPU. Just plug it into your router. I have had one now for several years.

Understood, but what're the options later on, when you might like to change
how you use and need the NAS? Or for that matter, if you desire a different configuration?
And: did you install your own drives, or purchase a ready-to-go NAS box?
There's certainly a large price difference between pre-loaded and empty NAS devices.

I don't see that it's possible to reconfigure or alter a dedicated NAS in ways that JBOD
can, without going into flashing firmware(if even possible) and the like.
They can be stacked and chained, and drives swapped to different capacities,
but that seems about all that can be done with them as your needs or desires may change.

SkedAddled wrote in post #18959829
Understood, but what're the options later on, when you might like to change
how you use and need the NAS? Or for that matter, if you desire a different configuration?
And: did you install your own drives, or purchase a ready-to-go NAS box?
There's certainly a large price difference between pre-loaded and empty NAS devices.

I don't see that it's possible to reconfigure or alter a dedicated NAS in ways that JBOD
can, without going into flashing firmware(if even possible) and the like.
They can be stacked and chained, and drives swapped to different capacities,
but that seems about all that can be done with them as your needs or desires may change.

I can do the following:

• Plug in additional NAS (both RAID or not-RAID) into the home network, to expand data storage
  
• Insert larger capacity pair of harddrives into the Synology to expand data storage (with the added complexity of getting already-stored data onto the new harddrives)
  

I agree that a local solution is always faster, but 3Mbps download/512Kbps upload may have been the average speed back in 2005 but even way back in 2015 the average speed was at least 15x that and in some cases more. I would think that anyone with cable these days the more common is 100mbps to 200.

gjl711 wrote in post #18959851
I agree that a local solution is always faster, but 3Mbps download/512Kbps upload may have been the average speed back in 2005 but even way back in 2015 the average speed was at least 15x that and in some cases more. I would think that anyone with cable these days the more common is 100mbps to 200.

That article which I linked was originally written in Jan 2016

I just ran an internet speed test on two computers, both on cable internet at 5pm (before most folks get home)

• Win 7 laptop via wireless connection to router, download 8mb/s
  
• Win 10 desktop via hardwire connection to router, download 190mb/s
  

Thanks for the great feedback everyone. A few questions since I'm new to all this. The NAS system you're referring to is what the guy at Microcenter what trying to talk me into with a network box connecting to my router which holds any number of drives? And that system configured in a RAID will provide one level of backup? I realize that the NAS system Craig is talking about may offer more flexibility, but at a relatively significant cost and I'm not clear on what benefit that's providing me. In addition and if I'm paranoid, I could/should purchase a cloud backup account as an offsite redundancy? Does that about sum it up? BTW, my internet speed is a 1000MBPS and I"m getting upload speeds of close to 800 MBPS, so time for uploading/downloading isn't near what you guys were mentioning. I hear you about the chance of these companies going under, but the odds of having a crash at the same time they are gone are fairly remote.

J-Blake wrote in post #18960164
Thanks for the great feedback everyone. A few questions since I'm new to all this. The NAS system you're referring to is what the guy at Microcenter what trying to talk me into with a network box connecting to my router which holds any number of drives? And that system configured in a RAID will provide one level of backup? I realize that the NAS system Craig is talking about may offer more flexibility, but at a relatively significant cost and I'm not clear on what benefit that's providing me. In addition and if I'm paranoid, I could/should purchase a cloud backup account as an offsite redundancy? Does that about sum it up? BTW, my internet speed is a 1000MBPS and I"m getting upload speeds of close to 800 MBPS, so time for uploading/downloading isn't near what you guys were mentioning. I hear you about the chance of these companies going under, but the odds of having a crash at the same time they are gone are fairly remote.

I wrote about a Synology unit and a Lacie unit, and both connect via internet cable to a router as the hub of a home network. There are capabilities to access the network data from anywhere, via the right security access.

RAID units typically allow choice of type of RAID configuration. RAID 1 is what provides one disk for storage, and the second disk is a mirrored copy of the other disk. If one disk fails, you get notified. Install a new disk and it automatically gets rebuilt to be a mirrored copy of the other (good) disk.

If you are really paranoid, you could simply do what I do...plug in a USB drive and copy all the contents of the RAID onto it (I just do incremental copies of changes/additions since the last time) and store it at a friend or relative or even at work, rather than use the cloud...too painfully slow to get data from the cloud! And as for a cloud company getting out of the business, if they do get out, then you will have to find a new service vendor and take many more days to move a backup copy of data to the new service!

You have tried an internet speed test and gotten real 800mb/sec results?! Wow. I have Xfinity for cable and internet, and their service only claims to offer "Download speeds up to 175 Mbps", which is about what I get during testing.

I thought the system the salesperson mentioned was a RAID 5? He described it as a 4 drive system with 25% of each drive dedicated as a backup, so if I configured it with 4 4TB drives it would have 12TB of storage plus a backup copy spread among the 4 drives. If one drive fails, the data and backup occurs in another location and is replaced. That was how it was explained to me, but as I type this I'm not sure the partition sizes would work. Maybe instead of 12 it would be closer to 9TB's, or whatever the proportion would be needed for the compressed backup.

I don't like the idea of manually copying. My system now is similar, but automated with Karen's Replicator which works flawlessly. I just need more space and am therefore looking for new ideas.

I have tested the internet speeds with several testing platforms and they all test out about the speed mentioned. I haven't put it to a real world test yet, but will as soon as I figure out my backup plan.

J-Blake wrote in post #18960296
I thought the system the salesperson mentioned was a RAID 5? He described it as a 4 drive system with 25% of each drive dedicated as a backup, so if I configured it with 4 4TB drives it would have 12TB of storage plus a backup copy spread among the 4 drives. If one drive fails, the data and backup occurs in another location and is replaced. That was how it was explained to me, but as I type this I'm not sure the partition sizes would work. Maybe instead of 12 it would be closer to 9TB's, or whatever the proportion would be needed for the compressed backup.

I don't like the idea of manually copying. My system now is similar, but automated with Karen's Replicator which works flawlessly. I just need more space and am therefore looking for new ideas.

I have tested the internet speeds with several testing platforms and they all test out about the speed mentioned. I haven't put it to a real world test yet, but will as soon as I figure out my backup plan.

That's not how raid 5 works. It does not store a backup copy of anything. If really interested search it and there are quite a few good descriptions but in a nutshell, 25% of each disk strategically places is dedicated to storing a calculated (XORed) value of the other three disks. RAID 5 is a decent 1 disk failure, fault tolerant system that gives a good balance between space and speed. A 2 disk failure kills all 4 though.

Thanks JJ! I've read up on this and have at least a third grade understanding of how RAID's work now. Since my goal is to expand my data storage in an efficient (cost efficient) way and in the process evaluate and possibly increase my data protection level is going with a Raid 5 setup any better than adding a NAS with a couple drives and using them as duplicate backup's in the same way I'm doing today? I'd change internal backup HD to be a data drive instead of a backup and have Karen's replicator manually duplicate the NAS drives to my internal HD's?

Is there a I/O speed difference for data storage as opposed to the internal HD setup I currently have? I'm using Google Wifi as a router if that's needed. If not, I could turn around my example above so that my NAS is my main data storage and use my internal drives are backup By doing this I could access my main data storage throughout my house. If there's not a big hit on performance (access) this seems like a good way to go. Or am I missing something else?

A bit off the topic of RAID configurations, the Inland Premium SSD 1TB in the M.2 2280 form factor is on sale at Micro Center (you have one in Denver) for $100. See https://www.microcente​r.com …nternal-solid-state-drive for the details.

With a usb 3.1 enclosure that would be a super fast backup drive.