Approve the Cookies
This website uses cookies to improve your user experience. By using this site, you agree to our use of cookies and our Privacy Policy.
OK
Index  •   • New posts  •   • RTAT  •   • 'Best of'  •   • Gallery  •   • Gear  •   • Reviews
Guest
New posts  •   • RTAT  •   • 'Best of'  •   • Gallery  •   • Gear  •   • Reviews
Register to forums    Log in

 
FORUMS General Gear Talk Data Storage, Memory Cards & Backup 
Thread started 02 Aug 2016 (Tuesday) 00:31
Search threadPrev/next
sponsored links
(this ad will go away when you log in as a registered member)

Best Simple Way to Backup for Wedding Photographer

 
Tigerkn
Goldmember
4,055 posts
Likes: 154
Joined Feb 2009
Location: CA
Post edited over 1 year ago by Tigerkn.
     
Aug 02, 2016 00:31 |  #1

Please help me to know what is the best / simple way to backup photos so that if one HD die and still doesn't affect the workflow (temporary) and/or losing any file.

I've heard of NAS, RAID 1, etc but they are all new to me. Please help me to know more! Which method is manageable as a DIY task? I can pay but rather do it myself so that I know how to maintain it and replace HD when it out of order.

My PC is a Windows 10, Intel(R) Core(TM) I7-4790 CPU @ 3.60GHz, RAM 16.0 GB, 64-bit OS, x64-based processor.

Current HD - 256GB SSD for OS and Apps - 2TBs WD Black for storage/working drive.

Thank you for your help in advance!


Website (external link) | Facebook (external link) | Instagram (external link) | Gears (external link)

  
  LOG IN TO REPLY
sponsored links
(this ad will go away when you log in as a registered member)
mike_d
Cream of the Crop
Avatar
5,093 posts
Likes: 373
Joined Aug 2009
     
Aug 02, 2016 00:47 |  #2

What post-processing software do you use? Lightroom can save your images to a backup hard drive automatically during import. So your setup could be as simple as two big hard drives: One working copy, one backup.

I personally use a Synology NAS. Everything lives on the NAS. One of the 5 drives inside can die and the contents are still accessible. It will beep loudly and email me if a drive fails. When a replacement drive is installed, it repairs the volume, all without any down time. The cheaper 2 bay NASs work similarly, but you must power down the unit to replace a drive. Since RAID is NOT a backup, I alternately connect 1 of 2 USB hard drives to the NAS to receive automatic nightly backups. I work from home so the "off" drive stays in a fire resistant box.

Don't forget off-site backup. You need at least one copy of your data somewhere else to guard against fire, flood, theft, or other disaster which would wipe out everything at your home/office. I use Crashplan for this. If you have a separate home and office, you could rotate your backups between the two locations.




  
  LOG IN TO REPLY
Tigerkn
THREAD ­ STARTER
Goldmember
4,055 posts
Likes: 154
Joined Feb 2009
Location: CA
     
Aug 02, 2016 00:55 |  #3

Yes, I use Lightroom. Sounds like what you are doing is very awesome. I would love to learn and copy your method. Thanks Mike! I will look into the NAS as I just read more and agree that RAID is not a backup.


Website (external link) | Facebook (external link) | Instagram (external link) | Gears (external link)

  
  LOG IN TO REPLY
Tigerkn
THREAD ­ STARTER
Goldmember
4,055 posts
Likes: 154
Joined Feb 2009
Location: CA
     
Aug 02, 2016 01:02 |  #4

Please help me to know the best bang for my bucks on a 2-bay NAS. Thanks!


Website (external link) | Facebook (external link) | Instagram (external link) | Gears (external link)

  
  LOG IN TO REPLY
MalVeauX
"Looks rough and well used"
Avatar
12,708 posts
Gallery: 1101 photos
Best ofs: 3
Likes: 8045
Joined Feb 2013
Location: Florida
Post edited over 1 year ago by MalVeauX.
     
Aug 02, 2016 01:03 |  #5

Heya,

I'm no expert on this, but I do have experience with mass storage and backups and maintain large libraries and having redundancy and avoiding down time, so maybe I can at least get you started in some directions to do some more research in.

First you need to figure out what level of redundancy you need. You also need to know how things actually work. For example, you mention if a HD failed, it not affecting current workflow, but that's only even an option if the HD that failed was the drive the OS was working from and/or the source drive of the media being edited and saved to, outside of RAM. If one of those fail, you will lose current work. Saved work will be fine likely. But current work sitting in RAM will dump and be lost. These systems are not designed to have physical drive failure and dump memory to ... the failed drive for safe keeping? See what I'm saying? Kind of is one of those problems. Just like if the power went out mid-edit and you were not on a backup battery. Whatever was in RAM at the same is lost. Saved files are fine though. Something to think about.

NAS is network attached storage. I wouldn't stress this right now. It's not a backup really. It's just another way to work. A NAS is just a way to have storage over a network, so that you can avoid messing around in your working computer, or have a place to store data and/or even work from that is accessible by multiple machines/users if needed. You can make a NAS complicated by adding redundancy to it, but really, again I would not push into this. You can build for it and still fail and still fail to rebuild the array and still lose data and have that down time and no data at the end of the day. A multi-drive NAS with redundancy and no down time sounds dreamy, but it's far from perfect. It's not a replacement for an external physical copy off site, or external digital copy like cloud storage off site.

RAID is just an old, old way to take small hard drives that are not expensive and string them together and use them as a single medium. It's risky. Even with redundancy, like RAID 1 (mirror, 1:1 copies), you can still have total failure and lose data (as it's in the same machine, on the same controller, doing the same tasks at the same time). And going into more dynamic RAID arrays, like 5, isn't really worth it either now, as it's even more of a chore to rebuild if there is failure and you still can easily lose the whole array of data. I would just avoid the idea of RAID all together.

Cloud storage is modern. It's not terribly fast, well, for everyone. But it's a great way to avoid total loss and is off the physical site. That's a good property of a good backup. It's not the only way to go, it's just another tool to add to the mix so that you have a diverse web of redundancy so that if one fails, you have others keeping some data intact. This is where you keep important current data that you cannot afford to lose. This is not where you keep vast libraries of 8~10 year old wedding RAWs that have not been culled.

If you really want redundancy, I would just have separate physical copies at different physical locations and/or with a cloud service too. It's not bullet proof, but there's no rebuild times, no special equipment, no real maintenance other than keeping a schedule to backup to all locations, rotate drives to keep healthy new drives in service, and learning a really important thing: culling data. You don't need to save RAWs from a wedding you did 8 years ago. It's done. Gone. Move on. Delete the data. You have small final JPGs that you can reference if needed, forever, somewhere on the web if you wish. You don't need to invest money into harboring ancient data that serves no purpose. This is for work, for weddings, not cataloging critical data. Try to keep the need in line with the reality of the usefulness of having the data for a long time, ie, more than 3~4 years.

Instead, I would focus on limiting potential down time. For example, if your main editing machine's HD/SSD with the OS on it failed, how do you respond? Do you replace it, reinstall all the software again all over on a new one and get back up and running? That takes a lot of time. Instead, if you had imaged the hard drive or SSD after you had it setup the way you wanted for editing, you could then easily just apply the image to a new drive in a few minutes or an hour or so, turn it on, and be right back up and running without having to configure or do anything. This is a smart way to backup your working machine. Hard drive cloning or imaging basically. It has nothing to do with the wedding images though, and everything to do with keeping your editing machine up and running with minimal down time in case of a HD/SSD fault.

Keep working medium off of the hard drive that your OS is on. That way if the medium drive fails, your working OS machine is fine; likewise, if your OS drive fails, your medium is intact on another drive.

Consider not using hard drives at all in the working machine and for when working on. Hard drives can be good as massive backup tools for physical copies. But hard drives are physical drives that move and have spinning parts, get hot, have heads inside, etc. Instead, consider using higher capacity SSD's that do not have moving parts. Less likely to have catastrophic mechanical failure. Plus they're much, much faster which supports your work flow.

Remember that if your electricity goes out, you can lose work. Everything goes down. Has to come back up and get back to work. Having uninterrupted power supplies can be a life saver and keep your uptime up, or when it's a total black out, at least let you save your current work and shut down and wait it out.

Lastly, having a 2nd machine is always a good idea. It doesn't have to be top shelf. Just something that will do some work in a pinch, while your main machine is down. If editing is how you make a living, the last thing you want to have to do is stop editing, leave, purchase a new system or get new parts, take the time to work on it, etc, losing lots of work time on a deadline. Weddings are not something to toy with after all. Having a 2nd machine, nothing fancy, a cheap beater, is a good way to keep working at least to some capacity while your down time on your main machine isn't costing you a client or income.

Hope that gets you started on where to read next.

Very best,


My Flickr (external link) :: My Astrobin (external link)

  
  LOG IN TO REPLY
Tigerkn
THREAD ­ STARTER
Goldmember
4,055 posts
Likes: 154
Joined Feb 2009
Location: CA
     
Aug 02, 2016 01:38 |  #6

Thank you Martin for your 1200 words of advice. I read it once and will read it again. Very details and very awesome!

Please let me know if I understand your advice correctly:
One main HD/SSD for the OS
One working HD/SSD for recent LR Catalogs/weddings
One storage HD
Frequence MANUAL backup to an external HD and be smart about what to backup and what not...

My method in the last several years is similar except I use the same drive as working drive / storage drive. Recently, one of the working drive / storage drive stopped working, I did not loose anything because I still have CF, SD cards and my manual backup. It just costs me time to re-create the LR catalog and import the files again.


Website (external link) | Facebook (external link) | Instagram (external link) | Gears (external link)

  
  LOG IN TO REPLY
MalVeauX
"Looks rough and well used"
Avatar
12,708 posts
Gallery: 1101 photos
Best ofs: 3
Likes: 8045
Joined Feb 2013
Location: Florida
Post edited over 1 year ago by MalVeauX.
     
Aug 02, 2016 01:53 |  #7

Heya,

Not the best way, just a way:

One large capacity SSD for the OS
One large capacity SSD for the working medium (LR catalogs, wedding images, etc)
* In your work flow, cull early, drop the non-keepers early, so you're not wasting time/space saving RAWs that are not keepers
An external storage HD source for (this can be USB, NAS, etc; dump current and important work to it daily and put it in a safe box) the different shoots
^ This again, but for the longer term storage, like the 3~4 year work. Beyond that, cull it.
Over night, upload current & important data to a cloud (I would not keep long term data here, since it's to be culled after a period anyways)

Image/clone your OS SSD so that you can replace it instantly if it failed. Having another SSD ready to be inserted as the replacement is a good idea. Inexpensive insurance to prevent more down time.
Having a 2nd working machine, a cheap one, something that just works, lets you continue working while you manage a failure on the other machine so you don't lose precious time.

Very best,


My Flickr (external link) :: My Astrobin (external link)

  
  LOG IN TO REPLY
drmaxx
Senior Member
920 posts
Gallery: 14 photos
Likes: 177
Joined Jul 2010
     
Aug 02, 2016 02:42 |  #8

Awesome post MalVeauX!

Just to make things even more complicated: You really should think carefully about the way you back up your important work. One of the currently most dangerous threat to your working files is probably not hardware failure but malware - especially ransomware. Right now, I get about 1 - 2 very targeted emails a week with malicious payload attached to it. Many of them are really well crafted, relating to my work and addressing me personally. The really mean and evil programs encrypt not only your local files, but also the drives connected to your computer. Additionally, some of them take some time to do their dirty work - which might mean that you will overwrite your duplicates with bad files.

If you are thinking about a proper file and backup management you might as well turn on your paranoia mode and think worst case. Then introduce a routine and stick to it. I for myself have three independent (physical) places where I mirror my important working files alternating over the time span of about 10 days. In addition, I have a proper backup (on a NAS) with daily incremental backups that go back for about a month. Archive files are offsite (on separate drives that are not on-line). For me cloud services are not worth the costs and the time it takes to up- and download.


Donate if you love POTN

  
  LOG IN TO REPLY
tim
Light Bringer
Avatar
50,920 posts
Likes: 333
Joined Nov 2004
Location: Wellington, New Zealand
     
Aug 03, 2016 15:23 |  #9

I considered a NAS, but decided against it because they're slower than internal drives if they go across a gigabit network, and often have proprietary formats.

Windows 10 supports two key technologies, ReFS and Storage Spaces. ReFS (like ZFS) is a file system that can check for and correct errors. Storage Spaces is software RAID. I suggest that you put to large drives (HGST (external link) are very reliable), format them ReFS and ensure you have integrity checking turned on, and then create a storage space with mirroring turned on. This setup will cost you relatively little, and will give you a reliable, redundant storage system inside your computer, with no extra wires, cords, or power supplies.

Other a third drive, maybe slightly larger than the internal pair, for backups. Run a piece of backup software that does INCREMENTAL backups, not mirroring - I identify some in a windows backup software review (external link) I did. I use two different backup products, one for compressible data, one for raw images.

Once images are processed I also generate medium res, medium quality jpeg files and upload them to Amazon S3 in the infrequent access storage class (ie "the cloud"). I only started doing this recently, I have four weddings up there and it's costing me $0.10 per month. I'm an AWS certified solution architect so this is easy for me, and most people could probably work it out, but it's not super user friendly.

I also have CrashPlan running on my PC, so anything I put on there is backed up. Since I have 100Mbps fiber internet it works pretty quickly.

I also run a Samsung 850 SSD (external link) for my OS. 120GB is plenty big enough for me, but 250 isn't much more expensive.

This is on the paranoid side of backups, but I don't like to lose data. I think in 10 years of weddings I've lost one image in the early days, and I don't know how that happened - user error I think. For context, I design enterprise solutions in government and large corporates, including hardware software and networks, where performance, reliability and security are crucial.

MalVeauX advice, while well meaning, isn't all completely accurate - he does say he's not an expert. For example RAID is designed to allow you to keep working during a drive failure, it wasn't designed to "take small hard drives that are not expensive and string them together and use them as a single medium". The whole point of RAID is to reduce risk. His advice to use SSDs and not spinning disks is possible, but expensive, and not completely practical. A UPS can be helpful to let you shut down cleanly, but using ReFS you don't need it as such, the file system will recover and you should have backups. A UPS isn't a bad idea though, but mine is sitting in the corner not plugged in. They take a bit of power even when turned off. I don't bother with a second computer, because I can buy one in two hours, and a couple of days downtime isn't really that important to me, but for high volume people sure that's reasonable. Having a spare OS disk isn't a bad idea - I keep OS images and happen to have a spare SSD, but I don't mirror to it regularly. Long term cloud storage is almost practical for many photographers, both in terms of cost and data sizes, but I don't bother storing RAWs in the cloud. However much of his advice is good, around keeping multiple copies in multiple locations.


Professional wedding photographer, solution architect and general technical guy with multiple Amazon Web Services certifications.
Read all my FAQs (wedding, printing, lighting, books, etc)

  
  LOG IN TO REPLY
mike_d
Cream of the Crop
Avatar
5,093 posts
Likes: 373
Joined Aug 2009
     
Aug 03, 2016 15:55 |  #10

tim wrote in post #18085640 (external link)
I considered a NAS, but decided against it because they're slower than internal drives if they go across a gigabit network, and often have proprietary formats..

My 5 bay Synology will saturate Gigabit Ethernet on sequential transfers which is just about the same performance as a single local SATA spinning drive. The NAS actually does far better on 4k transfers and simultaneous transfers than a single local drive. True, you can't compare it to a local SSD, but it compares favorably to a single hard drive which is what most people would use instead. I spent a fair bit of coin to get the performance though. Most NASs are no faster than a USB 2.0 drive.




  
  LOG IN TO REPLY
Tigerkn
THREAD ­ STARTER
Goldmember
4,055 posts
Likes: 154
Joined Feb 2009
Location: CA
     
Aug 03, 2016 16:20 |  #11

Thank you Tim! Mike, Martin and Tim, why are you guys not my next door neighbor? :)
I used to enjoy working on PC but not anymore. Photography is too much fun and nothing else is matter. I'll read this thread again a few times and do something before my PC get sick again.

Thank you again guys!


Website (external link) | Facebook (external link) | Instagram (external link) | Gears (external link)

  
  LOG IN TO REPLY
tim
Light Bringer
Avatar
50,920 posts
Likes: 333
Joined Nov 2004
Location: Wellington, New Zealand
     
Aug 03, 2016 16:27 |  #12

mike_d wrote in post #18085656 (external link)
My 5 bay Synology will saturate Gigabit Ethernet on sequential transfers which is just about the same performance as a single local SATA spinning drive. The NAS actually does far better on 4k transfers and simultaneous transfers than a single local drive. True, you can't compare it to a local SSD, but it compares favorably to a single hard drive which is what most people would use instead. I spent a fair bit of coin to get the performance though. Most NASs are no faster than a USB 2.0 drive.

Yeah, the price to get a decent fast NAS can be quite high. I figured my computer is on most of the time anyway, and W10 has all these great features (it's why I upgraded to it), so I should save money and use them. I also wanted ZFS or some other reliable file system, which further increases cost. I was keen on those features as my computer had bad RAM and I had a little bit of file corruption, which tbh reliable disks wouldn't have really helped, I don't think.

1Gbps is 112MBps, so fast hard drives throughput which peaks around 150MBps will be reduced slightly. There will also be more latency, but I don't know how much - probably not much. Keeping cache on the local SSD and RAW files on the NAS probably means it wouldn't matter that much. My working set of RAW files fits in RAM so once loaded they don't hit the disk again anyway - if NAS files are cached.

Really, putting a small computer in place just to store data for a big computer, which is quite capable of doing this itself, isn't sensible in all cases. I have multiple devices accessing the system from around the house, the computer does the job well. In other cases a NAS may make sense. But as has been said earlier, NAS and RAID are not backups, they're to help data more accessible and perhaps survive one disk failure. Incremental offsite backups are essential for important data.


Professional wedding photographer, solution architect and general technical guy with multiple Amazon Web Services certifications.
Read all my FAQs (wedding, printing, lighting, books, etc)

  
  LOG IN TO REPLY
Charlie
Guess What! I'm Pregnant!
14,584 posts
Gallery: 8 photos
Likes: 4869
Joined Sep 2007
Post edited over 1 year ago by Charlie. (2 edits in all)
     
Aug 03, 2016 19:06 |  #13

raid is not a backup, it's to prevent downtime.

if you want simple, here it is:

Main PC - > work drive, however big that may be.

USB backup - > same size as work drive or bigger

Amazon backup or equivalent as secondary backup. Runs in the background, unlimited I think. Software is slick, can access by phone or desktop app to restore.

btw, the advantage of NAS is to share with other devices on your network while your main pc is OFF. If you dont do that, then keep it simple with USB backups.


Sony A7rii/A7riii - FE 12-24/4 - FE 24-240 - FE 28/2 - FE 35/2.8 - FE 50/1.8 - FE 85/1.8 - EF 135/1.8 Art - F 600/5.6 - CZ 35-70, 100-300 - Astro Rok 14/2.8, 24/1.4 - Tamron 28-75 f2.8, 70-200 f2.8 VC

  
  LOG IN TO REPLY
photopat
Senior Member
569 posts
Likes: 1
Joined Jan 2010
Location: South Florida, USA
     
Aug 30, 2016 17:08 as a reply to  @ Charlie's post |  #14

On my setup I opted for the NAS as primary storage (different versions of Synology over the years) with 4+ drives with RAID.
I edit directly from the NAS files. My computer and NAS have UPS.

In case of computer crash, I can still access my data rapidly from another computer.
The RAID protect against drive failure.
I don't have to keep my computer on all the time for backup (but the NAS is always on).
I can remotely log into my NAS to access files I may need away from home.

As I was not happy with the commercial online backup solutions (speed and cost), I installed another NAS in another remote location and the backup are done automatically in the background from one NAS to the other every night. I also, every other few months, make an additional backup on a hard drive that I store in a bank safe.


toys list

  
  LOG IN TO REPLY
Tigerkn
THREAD ­ STARTER
Goldmember
4,055 posts
Likes: 154
Joined Feb 2009
Location: CA
Post edited over 1 year ago by Tigerkn.
     
Sep 18, 2016 14:18 |  #15

Is it safe, practical to use Window 10 feature Storage Space (Mirror) as a working drive/s for LR Catalogs?


Website (external link) | Facebook (external link) | Instagram (external link) | Gears (external link)

  
  LOG IN TO REPLY
sponsored links
(this ad will go away when you log in as a registered member)

5,699 views & 7 likes for this thread
Best Simple Way to Backup for Wedding Photographer
FORUMS General Gear Talk Data Storage, Memory Cards & Backup 
AAA
x 1600
y 1600

Jump to forum...   •  Rules   •  Index   •  New posts   •  RTAT   •  'Best of'   •  Gallery   •  Gear   •  Reviews   •  Member list   •  Polls   •  Image rules   •  Search   •  Password reset

Not a member yet?
Register to forums
Registered members may log in to forums and access all the features: full search, image upload, follow forums, own gear list and ratings, likes, more forums, private messaging, thread follow, notifications, own gallery, all settings, view hosted photos, own reviews, see more and do more... and all is free. Don't be a stranger - register now and start posting!


COOKIES DISCLAIMER: This website uses cookies to improve your user experience. By using this site, you agree to our use of cookies and to our privacy policy.
Privacy policy and cookie usage info.


POWERED BY AMASS forum software 2.1forum software
version 2.1 /
code and design
by Pekka Saarinen ©
for photography-on-the.net

Latest registered member is kiwix
911 guests, 373 members online
Simultaneous users record so far is 6430, that happened on Dec 03, 2017

Photography-on-the.net Digital Photography Forums is the website for photographers and all who love great photos, camera and post processing techniques, gear talk, discussion and sharing. Professionals, hobbyists, newbies and those who don't even own a camera -- all are welcome regardless of skill, favourite brand, gear, gender or age. Registering and usage is free.