Approve the Cookies
This website uses cookies to improve your user experience. By using this site, you agree to our use of cookies and our Privacy Policy.
OK
Forums  •   • New posts  •   • RTAT  •   • 'Best of'  •   • Gallery  •   • Gear
Guest
Forums  •   • New posts  •   • RTAT  •   • 'Best of'  •   • Gallery  •   • Gear
Register to forums    Log in

 
FORUMS Post Processing, Marketing & Presenting Photos RAW, Post Processing & Printing 
Thread started 24 Jan 2011 (Monday) 03:12
Search threadPrev/next
sponsored links (only for non-logged)

Please C&C my new backup workflow

 
TweakMDS
Goldmember
Avatar
2,242 posts
Likes: 1
Joined Nov 2008
Location: Netherlands
     
Jan 24, 2011 03:12 |  #1

I recently decided to professionalize my backup (and recovery) workflow for digital photography. Even though I'm just a hobbyist, I come from an IT background and decided a professional workflow is not very hard or expensive to realize for the end-user. Maybe some steps are overkill, but I hope it can be of some help to other people who don't have much experience in this area.

Hardware and media prerequisites:
- Cardreader (can of course also be camera usb connection)
- Computer harddisk (single disk, no raid)
- Synology NAS (single disk, no raid - stored in the basement)
- External 2.5" USB harddrive
- DVD burner
- blank DVD media brand A
- blank DVD media brand B
Software prerequisites:
- Adobe Lightroom
- Synology Data Replicator 3
- 7-zip
- QuickPar

Stage 1 - Import (frequency: manual):
Reading the CF card into my lightroom catalog.
CR2 files get stored on my local disk.
Lightroom catalog is updated on local disk

Stage 2 - Workstation backup (frequency: every 4 hours):
Lightroom Catalog and files on disk are backup up to Synology NAS.
Synology Data Replicator is used for this on file level.
Lightroom also creates a backup of the catalog on my OS disk which is also backed up in this stage.

Stage 3 - Server backup (frequency: once a week on sunday at 0:00)
Synology backup storage is backed up to the USB disk connected to the NAS. The USB disk has to be connected manually (or left connected).

Stage 4 - Archive (frequency: every year).
This stage is slightly irregular; I manually generated subdirectories of 4.5 GB each, place these in subfolders per 4 (labelled A-Z) and add 25% of PAR2 files (using quickpar) to this. This gives me a set of 5 DVD's for every 18 GB (A1 to A5, B1 - B5 etc). If one of the four data DVDs fails completely, it can be recovered using the parity disk.
Since my current backup spanned a total of around 100 GB, this gave me 30 DVD disks in total. I stored these in a spindle in my parents' basement.
Still not 100% decided on the last step since it was VERY time consuming, but then again, this was 2 1/2 years worth of images (2008 - 2010) stored in one go. For the next backup run I'll just do the increment of everything after 2010 only.

After the archive step I created a new lightroom catalog for 2011. I removed the old catalog from the backup process, but it's still present on my local drive and on the USB drive (as well as on the DVD's).

Discussion
My imaging process (from camera to post processing) has a few single points of failure. Since I don't own a camera with dual card slots, the CF card in my body is basically a high risk area. For now this is an acceptable risk to me.
I also only have one body, so if anything goes wrong during the shooting, I'm also screwed - although for weddings I borrow or rent an additional body.
In the event of fire or theft, I spread the risk by putting the NAS in another room, but in a heavy fire it's likely to still die too. I considered this and decided I'd have bigger problems then. The archive is likely to be safe.
I might script the archive step to automatically span dvd files and generate a par2 for every 18GB in the future. It doesn't necessarily need to be every year, can also be every 18Gb of CR2 data.

Pros
- Short term, medium term and long term backup process.
- Low cost (I already had the NAS, USB disks are around €100 for like 1TB - easily worth it to me).
- Data is stored relatively securely.
- DVD's have a 1 in 5 redundancy.
- A NAS is great to work with in backup and recovery.

Cons
- A few single points of failure (in-camera, CF card readout, NAS and workstation are in the same building, USB disk is usually also in the same building)
- Eats up a LOT of storage on my NAS (around 180GB with max 3 versions of a file).
- This is quit a complex process
- The archive stage is VERY time consuming, but it does give me the option to clean up.
- Burning 30 DVD's at 4x speed with verify sucks.


Feel free to leave any comments or suggestions :)


Some of my lenses focus beyond infinity...!
~Michael
Gear | Flickr (external link)
"My featured shots" (external link)

  
  LOG IN TO REPLY
ZXDrew
Goldmember
1,027 posts
Joined Mar 2010
Location: Austin, TX
     
Jan 24, 2011 12:36 |  #2

Interesting back up process. I make a backup copy of all my images onto my 2nd internal hard drive when I import them using the backup feature in lightroom. I also keep a backup copy of catalog on the 2nd drive. Every event I backup one of the two portable drives I have and place it in a secure off site location. Once I place one, I grab the other and bring it home and store it in another random part of the house. It allows me to have one backup on my computer on a separate drive, and if I should need another off-site with a third older version in the house. I pickup the portable hard drives when they are on sale at Fry's. Every 6 months or so I run through anywhere from 250-500gb worth of photos. I plan on keeping my photos in raw format for two years before converting them to JPEG for archival purposes.

I've lost 1 drive and 0 photos so far. But to be fair the drive didn't fail just the laptop's GPU and boot sector.

It seems like the DVD's are overkill. If I'm not mistaken don't DVD's have a shelf life before they go bad?


PhotoWolfe.com (external link)
Facebook.com/PhotoWolfe (external link)
Gear / My flickr (external link)

  
  LOG IN TO REPLY
sponsored links (only for non-logged)

1,662 views & 0 likes for this thread, 2 members have posted to it.
Please C&C my new backup workflow
FORUMS Post Processing, Marketing & Presenting Photos RAW, Post Processing & Printing 
AAA
x 1600
y 1600

Jump to forum...   •  Rules   •  Forums   •  New posts   •  RTAT   •  'Best of'   •  Gallery   •  Gear   •  Reviews   •  Member list   •  Polls   •  Image rules   •  Search   •  Password reset   •  Home

Not a member yet?
Register to forums
Registered members may log in to forums and access all the features: full search, image upload, follow forums, own gear list and ratings, likes, more forums, private messaging, thread follow, notifications, own gallery, all settings, view hosted photos, own reviews, see more and do more... and all is free. Don't be a stranger - register now and start posting!


COOKIES DISCLAIMER: This website uses cookies to improve your user experience. By using this site, you agree to our use of cookies and to our privacy policy.
Privacy policy and cookie usage info.


POWERED BY AMASS forum software 2.58forum software
version 2.58 /
code and design
by Pekka Saarinen ©
for photography-on-the.net

Latest registered member is Monkeytoes
1487 guests, 187 members online
Simultaneous users record so far is 15,144, that happened on Nov 22, 2018

Photography-on-the.net Digital Photography Forums is the website for photographers and all who love great photos, camera and post processing techniques, gear talk, discussion and sharing. Professionals, hobbyists, newbies and those who don't even own a camera -- all are welcome regardless of skill, favourite brand, gear, gender or age. Registering and usage is free.