Approve the Cookies
This website uses cookies to improve your user experience. By using this site, you agree to our use of cookies and our Privacy Policy.
OK
Index  •   • New posts  •   • RTAT  •   • 'Best of'  •   • Gallery  •   • Gear  •   • Reviews
Guest
New posts  •   • RTAT  •   • 'Best of'  •   • Gallery  •   • Gear  •   • Reviews
Register to forums    Log in

 
FORUMS General Gear Talk Data Storage, Memory Cards & Backup 
Thread started 03 May 2017 (Wednesday) 16:17
Search threadPrev/next
sponsored links
(this ad will go away when you log in as a registered member)

back up file storage

 
lijoec
Goldmember
Avatar
1,029 posts
Gallery: 10 photos
Likes: 3868
Joined Jan 2009
Location: Long Island
     
May 03, 2017 16:17 |  #1

I'm not very tech savvy, All of my family and vacation shots I've been putting on CD's! this is so tedious and a pain in the butt to keep track of. is there a better way of doing this? I was thinking an external hard drive, but I'm thinking they can crash and break also, No?

thanks in Advance,
Joe


Cheers,
Joe
"Image Editing OK"

  
  LOG IN TO REPLY
sponsored links
(this ad will go away when you log in as a registered member)
Bassat
"I am still in my underwear."
8,075 posts
Likes: 2733
Joined Oct 2015
     
May 03, 2017 18:23 |  #2

lijoec wrote in post #18345508 (external link)
I'm not very tech savvy, All of my family and vacation shots I've been putting on CD's! this is so tedious and a pain in the butt to keep track of. is there a better way of doing this? I was thinking an external hard drive, but I'm thinking they can crash and break also, No?

thanks in Advance,
Joe

Yes, any mechano-electrical (or any other) device can fail. That is why my computer is set up with a back-up internal hard disk. I also use two external hard disks, and keep one of those in my car. On top of that, all my photos, and all my music, gets backed up a 4th time on yet another hard disk. Just so I can get up and running quickly in the event of a disk failure, I keep another full-size drive in the desk with the OS already installed. If my HDD fails, I can be up and running (minimally) in less than 5 minutes. In an afternoon, I can be fully restored. Disks are cheap; buy several.




  
  LOG IN TO REPLY
NBEast
Goldmember
Avatar
1,699 posts
Gallery: 11 photos
Likes: 67
Joined Aug 2005
Location: So Cal
     
May 03, 2017 18:38 |  #3

I upload the photos I care about to Smugmug. I'm grandfathered in at $35/yr for unlimited storage. Since I use them for file sharing anyway, it's a nice storage alternative. The bonus is that I don't have to worry about transferring stuff around when I switch PCs or switch to one of our laptops (which is often).

For the other photos and raw files, I guess I don't really care if they're lost so I just leave them on my giant hard drive. Once a year or so I copy them to an external but a thief would probably grab them both since they sit next to one another.


Gear List / Photos (external link)

  
  LOG IN TO REPLY
Jethr0
Goldmember
Avatar
1,050 posts
Gallery: 91 photos
Likes: 719
Joined Aug 2012
Location: Ottawa, Ontario, Canada
     
May 03, 2017 18:59 |  #4

I use iCloud to automatically mirror my document etc...I selectively copy keepers/sentimental pics. This is in addition to a local time capsule backup device. The 2 layers of protection is comforting.

Some kind of cheap cloud file service from a stable/well known provider and some kind of basic local backup would cover most of your basic needs.

If I were making a living @ photography I might do something different....but alas, I am not making a living that way.


www.jefflowe.ca (external link)
Instagram: https://www.instagram.com/jeff​lowe.ca (external link)

  
  LOG IN TO REPLY
sandpiper
Cream of the Crop
Avatar
7,171 posts
Gallery: 1 photo
Likes: 50
Joined Aug 2006
Location: Merseyside, England
     
May 03, 2017 19:11 |  #5

No matter what method you choose to store your files, they can (and probably eventually WILL) fail. Hard drives can fail, cloud storage companies can (and have in the past) disappear overnight, CDs and DVDs can degrade and become unreadable over time (archival quality discs minimise that risk but cost more than regular discs). RAID systems seem safe but multiple copies across the system don't help if the controller goes and you lose everything. And of course, even if you did have some amazing form of storage that was never going to simply fail, there are also the risks of disaster such as fire, flood etc., destroying your house or just a simple burglary where somebody steals your computer and the backup hardware.

So, the rule is multiple backups in multiple locations and check them from time to time for integrity.

Some people also say you should use more than one method, perhaps having a cloud storage setup as an off-site backup, that is a matter of personal preference. My personal choice is similar to Bassats, multiple external HDDs. After a shoot the cards are downloaded onto my main drive and my main on-site backup drive, as well as a portable drive, before the cards are formatted ready for my next shoot. That gives me three sets of the files from the shoot, one plugged into the computer, one that gets unplugged and tucked away in another room (in case of theft, a burglar is unlikely to find the hidden drive) the portable drive goes in my car and when I visit any of the 3 friends who keep an off-site backup drive of mine, any recent stuff is copied to their drive. As I edit the work it also goes through the same backup process, admittedly there might be a couple of evenings work sometimes sat only on the main drive until I get around to plugging the other drives in to backup, but worse case scenario is I have to redo a few hours work after restoring the original files from a backup. Nothing irreplaceable is lost, I always have the originals to fall back on. More often than not though, I plug one of the backups in while I work and backup edits as I do them

Once I have finished working on the shoot all originals and edits (and often stages of edits) will be stored on 6 drives (2 at home, 1 in the car and 3 scattered around at friends homes). In the event of my main drive failing it takes me a couple of minutes to dig out the backup drive, plug it in, and carry on.

As Bassat said, drives are cheap, sure one may fail from time to time (or, in my case, get knocked on the floor whilst running) but it takes almost no time to switch to a backup and carry on. Then, buy a new drive (or send it back for a free replacement under warranty) and transfer all your files over from the other drive.




  
  LOG IN TO REPLY
NBEast
Goldmember
Avatar
1,699 posts
Gallery: 11 photos
Likes: 67
Joined Aug 2005
Location: So Cal
     
May 05, 2017 14:46 |  #6

Another $.02; question keeping everything. Just keep the keepers. It'll make the whole archival process far more manageable.

For me; any raw file over a year old can be deleted. By then I've created my desired JPGs. That's 80% of the photo space right there. Plus the RAW tends to be everything were-as the JPGS are just the keepers.


Gear List / Photos (external link)

  
  LOG IN TO REPLY
CyberDyneSystems
Admin (type T-2000)
Avatar
49,509 posts
Gallery: 160 photos
Likes: 6075
Joined Apr 2003
Location: Rhode Island USA
     
May 05, 2017 15:11 |  #7

lijoec wrote in post #18345508 (external link)
...I was thinking an external hard drive, but I'm thinking they can crash and break also, No?

thanks in Advance,
Joe

Yes, but the solution is simply external hard drives plural.

They are so affordable these days, even the little 2.5" laptop sized ones.
Keep one attached to the desktop at all times,. do your monthly/weekly/or daily back up to it, at your prefered schedule.

Keep a second unplugged (or off) at all time except for back ups every few months or so,. AFTER you do your scheduled anti virus sweep on the system. Unplugged means no way to get virus or ransom ware on that one.

Use a 3rd for off site storage. (Maybe pay a little extra for a laptop sized?) ,. Keep this at work, or where ever as long as it's not at home with the rest. Every six months swap it with the 2nd hard drive above,.


GEAR LIST
CDS' HOT LINKS
Jake Hegnauer Photography (external link)

  
  LOG IN TO REPLY
CyberDyneSystems
Admin (type T-2000)
Avatar
49,509 posts
Gallery: 160 photos
Likes: 6075
Joined Apr 2003
Location: Rhode Island USA
Post edited over 1 year ago by CyberDyneSystems. (2 edits in all)
     
May 05, 2017 15:23 |  #8

NBEast wrote in post #18347237 (external link)
Another $.02; question keeping everything. Just keep the keepers. It'll make the whole archival process far more manageable.

For me; any raw file over a year old can be deleted. By then I've created my desired JPGs. That's 80% of the photo space right there. Plus the RAW tends to be everything were-as the JPGS are just the keepers.


This reminds of one thing I used to do, that I should really do again, (I think I stopped when I upgraded my workstation a few years ago and "forgot" to implement this)

- IF, like me you aren't the best at keeping up to date with culling ;)

- And I think this would REALLY be advantageous for cloud back up....

I had a batch file, using xcopy that would only copy jpeg, tif, and .psd. It does not duplicate the RAW files.

It would get copied to a different directory but maintain the exact same folder structure of the originals. This directory would - minus all the unculled RAWs be a fraction of the main photo volume, but when that gets backed up, it makes a more reasonably sized archive of all the images that I had spent time and energy processing.

I started doing this long before cloud storage made sense, and was using DVD-R.

I might start this practice again and see how large the resulting volume is. Might be just the ticket to take advantage of cloud storage. (no way will I ever back up my unculled RAWS to cloud)

this would be my "Hail Mary" compact back up.


GEAR LIST
CDS' HOT LINKS
Jake Hegnauer Photography (external link)

  
  LOG IN TO REPLY
John ­ from ­ PA
Cream of the Crop
8,715 posts
Likes: 671
Joined May 2003
Location: Southeast Pennsylvania
     
May 16, 2017 07:41 |  #9

CyberDyneSystems wrote in post #18347266 (external link)
I might start this practice again and see how large the resulting volume is. Might be just the ticket to take advantage of cloud storage. (no way will I ever back up my unculled RAWS to cloud)

this would be my "Hail Mary" compact back up.

One thing that should be considered is the Microsoft utility called RichCopy. It works well at least up to Win 7 and likely later. In some simple "subjective" comparisons I find it faster than Xcopy. If you aren't into writing batch files it can also simplify the task.

The utility is discussed and can be downloaded at https://technet.micros​oft.com ….04.utilityspot​light.aspx (external link). The link to download is at the end of the line (near the top) reading "Code download available at: HoffmanUtilitySpotligh​t2009_04.exe (5,896 KB)".




  
  LOG IN TO REPLY
CyberDyneSystems
Admin (type T-2000)
Avatar
49,509 posts
Gallery: 160 photos
Likes: 6075
Joined Apr 2003
Location: Rhode Island USA
     
May 16, 2017 12:02 |  #10

Thanks for that.

In the mean time, rewrote my xcopy bat file and ran it the other night.
I forgot though that since I last used it, I have a number of shoots that were jpeg only. So having tons of unedited jpegs makes the inclusion of jpeg in the xcopy less targeted towards only edited images, but at least jpegs are much smaller than unedited raw.


The original volume was in fact 1.8 TB

The Xcopy of tif, psd and jpeg was only 158 gigabytes.

Or less than 1/10th the size of the pile of unedited RAW files.

= I am way too lazy/bad/inefficient at culling!!!


GEAR LIST
CDS' HOT LINKS
Jake Hegnauer Photography (external link)

  
  LOG IN TO REPLY
CyberDyneSystems
Admin (type T-2000)
Avatar
49,509 posts
Gallery: 160 photos
Likes: 6075
Joined Apr 2003
Location: Rhode Island USA
     
May 16, 2017 12:13 |  #11

P.S. here's my .bat file.
Simple and to the point.
Sadly due to xcopy restrictions you can not specify more than one file type per line , so it's 3 lines.

:: tif
xcopy f:\digita~1\*.tif o:\tifbak\ /d/s/v

::jpeg
xcopy f:\digita~1\*.jpg o:\tifbak\ /d/s/v

::psd
xcopy f:\digita~1\*.psd o:\tifbak\ /d/s/v


Obviously one would have to edit the locations,.
The /s switch COULD be replaced by the /e (empty) switch, which would completely copy the entire directory structure from the source, but if those folders had only raw files, they would be empty folders, no no point IMHO. ith the /s switch, it does however reproduce exactly the source directory structure with the exception that it only does so if the directory has a tf, psd, or jpeg to copy. or my uses this works perfectly as my directory structure keeps the RAW and processed images in subfolders of the same higher level folder.


GEAR LIST
CDS' HOT LINKS
Jake Hegnauer Photography (external link)

  
  LOG IN TO REPLY
tim
Light Bringer
Avatar
50,962 posts
Likes: 351
Joined Nov 2004
Location: Wellington, New Zealand
     
May 16, 2017 15:13 |  #12

IMHO the key to backups are:

  • Incremental backups: this protects against ransomware / crytoware, which are increasingly common. There was a huge attack recently. Mirrors are not backups, they're copies.
  • Offline storage: anything connected to your computer is vulnerable. Anything in the same place as your data can be stolen or burned in a fire
  • Multiple copies: if data is important, keep multiple copies in multiple places, ideally in multiple formats

    Incremental / Mirroring
    A real key is incremental backups. If you simply copy or mirror your files regularly, and a virus gets in, you may overwrite your good files with bad before you realise. Incremental backups address this.

    Robocopy, xcopy, etc, have little place in backup systems IMHO, other than moving between internal disks and such.

    Online Backups
    Online backups are great, they're very convenient. However since there's a client on your PC there is access to those backups. It's possible that a clever virus could instruct the server to delete all your backups, close your account, etc. So don't rely on connected online backups.

    My System and software
    Here's what I do.

  • I keep two disk in my house that has a copy of everything important. One is current active data, the other is archives.
  • I keep two disks offsite that have the same data as above
  • I use CrashPlan to back many things up online: financials, family photos, etc. I don't store everything online, though these days that is practical. The more data you have stored the more resources it tends to require, if your backup system does de-duplication. This is because block hashes are typically kept in memory during a backup.
  • I use Duplicati (external link) to do incremental, deduplicated backups to my external disks and AWS S3. CloudBerry Backup (external link) is a simpler tool that's just as good. Different data goes to different places.
  • I keep 2500px Q8 jpeg versions of all my images stored in AWS Glacier, zipped by client or year. It's very cheap storage, and it's incredibly reliable... so long as you pay your bill. The files are something like 99.9999999% durable, so in practice you should never lose a file. Each file is stored in three different data centers, integrity is regularly checked, with bad blocks replaced from the other two copies.

  • Professional wedding photographer, solution architect and general technical guy with multiple Amazon Web Services certifications.
    Read all my FAQs (wedding, printing, lighting, books, etc)

      
      LOG IN TO REPLY
    sponsored links
    (this ad will go away when you log in as a registered member)

    1,852 views & 7 likes for this thread
    back up file storage
    FORUMS General Gear Talk Data Storage, Memory Cards & Backup 
    AAA
    x 1600
    y 1600

    Jump to forum...   •  Rules   •  Index   •  New posts   •  RTAT   •  'Best of'   •  Gallery   •  Gear   •  Reviews   •  Member list   •  Polls   •  Image rules   •  Search   •  Password reset

    Not a member yet?
    Register to forums
    Registered members may log in to forums and access all the features: full search, image upload, follow forums, own gear list and ratings, likes, more forums, private messaging, thread follow, notifications, own gallery, all settings, view hosted photos, own reviews, see more and do more... and all is free. Don't be a stranger - register now and start posting!


    COOKIES DISCLAIMER: This website uses cookies to improve your user experience. By using this site, you agree to our use of cookies and to our privacy policy.
    Privacy policy and cookie usage info.


    POWERED BY AMASS forum software 2.1forum software
    version 2.1 /
    code and design
    by Pekka Saarinen ©
    for photography-on-the.net

    Latest registered member is Gianpaolo
    1069 guests, 346 members online
    Simultaneous users record so far is 15144, that happened on Nov 22, 2018

    Photography-on-the.net Digital Photography Forums is the website for photographers and all who love great photos, camera and post processing techniques, gear talk, discussion and sharing. Professionals, hobbyists, newbies and those who don't even own a camera -- all are welcome regardless of skill, favourite brand, gear, gender or age. Registering and usage is free.