When was the last time you inspected your backups?

I meant to post this the other week when I realized that I was late for this months data inspection and audit, but then got distracted and never made a post.

Long story short: When was the last time you actually looked at the files your backup is creating, and evaluated the ways it could get deleted?

Doesn't seem as common, but I think the posts of people having issues with malformed backups ("Backups" were all on the same computer that went sparky-sparky, corrupted data, forgotten encryption passwords, automated services setup ages ago and no one noticed they stopped working sometime last year, etc.) are in a way even sadder than the panicked postings of someone trying to recover data when they had no backup plans in place.


If I selected all your images and deleted them on your computer now, then threw your computer in a fire, what are you sure you're going to have left?

Thanks. I just updated mine and took a look. All good here.

Now I need to work on a plan to get a copy offsite.

I don't do an automated backup. But I have copies of my raw files, among other things manually copied on two hard drives.
One at home and one at work. I also have backups of them on Flickr as Jpegs, should both hard drives fail, jpeg is better than no--peg

I do plug them in and boot them up at least once a month, usually for updating, but also good for checking.
Worst thing I could do is leave Drive B completely untouched.
Having Drive A fail or get lost, only to realize my 'sure thing' Drive B already broken, untouched for months.
Just a senerio, but a great reason to check on you drives, not just assume they will work after disaster strikes.

External drive back to NAS and to Crashplan. Crashplan has unlimited versioning. I'm good!

flowrider wrote in post #18020916
External drive back to NAS and to Crashplan. Crashplan has unlimited versioning. I'm good!

Yes, but what is Actually on your NAS and Crashplan? Is what you think actually there, or has some unnoticed error stopped anything from months ago being properly backed up?


Also another good point to remember: Can you decrypt you data that is backed up? One of my clients a few months ago got caught in an awkward position: Their backups were all working flawlessly, but their systems Admin had since moved on, and no one knew where any of the passwords for anything were. It was made extra awkward because their sysadmin hadn't simply moved on to another job, he had moved on to a completely new career, and was off grid doing research somewhere on the other side of the globe and no one had any clue how to contact him.


Data inspections are kind of like fire drills: Everyone knows that you exit the building when the fire alarm goes off, but running drills helps with things like people understanding and knowing what that weird annoying buzzing sound is, and reminds you to consider things like the effect of noise cancelling headphones,... Like the ones worn by the coworker who failed to even notice anyone had left the building one time.

Luckless wrote in post #18020932
Yes, but what is Actually on your NAS and Crashplan? Is what you think actually there, or has some unnoticed error stopped anything from months ago being properly backed up?

Also another good point to remember: Can you decrypt you data that is backed up? One of my clients a few months ago got caught in an awkward position: Their backups were all working flawlessly, but their systems Admin had since moved on, and no one knew where any of the passwords for anything were. It was made extra awkward because their sysadmin hadn't simply moved on to another job, he had moved on to a completely new career, and was off grid doing research somewhere on the other side of the globe and no one had any clue how to contact him.

Data inspections are kind of like fire drills: Everyone knows that you exit the building when the fire alarm goes off, but running drills helps with things like people understanding and knowing what that weird annoying buzzing sound is, and reminds you to consider things like the effect of noise cancelling headphones,... Like the ones worn by the coworker who failed to even notice anyone had left the building one time.

Crashplan emails weekly backup reports emails the user if any machine in their account has not been backed up in 3 days.

I am my own admin and I keep my passwords in LastPass so there's little chance I'd be unable to restore a backup. And yes, I do restore from Crashplan from time to time. I also have local backups.

I run the Disk Utility program to check all drives for errors, is that good enough?

For anything further, I'm honestly afraid to look. I have a 2015 rMBP and two external drives. On one, I just throw all my wedding RAW files and video stuff, and on the other, a duplicate copy of the wedding RAW files* and in a separate folder, run the Time Machine backup program. I keep the latter drive at my office locked up so its off site. I'm afraid of going in there and somehow screwing up by reverting my current system back to an earlier version by screwing around in time machine.

*I keep the raw files on the original SD/CF cards until I have the edited JPEGs uploaded to some sort of cloud based service. So I try to keep three copies of images for paid work at all times.

i do monthly backups to usb drive, i should probaably do more often

sirmium wrote in post #18031983
i do monthly backups to usb drive, i should probaably do more often

Yes, but what is actually on that USB drive right now? The point of this thread isn't just "Did you run your backup utility lately?", but rather "Have you checked that your backups are actually working, and contain the data you believe they do."

Having a backup is a good idea. However a backup of only the windows thumbnails files, or a backup encrypted with a 512bit process and no key, or a backup of your C:/Documents/Photos when you moved everything over to your D:/Documents/Photos, or C:/photos doesn't do you any good.

Luckless wrote in post #18032036
Yes, but what is actually on that USB drive right now? The point of this thread isn't just "Did you run your backup utility lately?", but rather "Have you checked that your backups are actually working, and contain the data you believe they do."

Having a backup is a good idea. However a backup of only the windows thumbnails files, or a backup encrypted with a 512bit process and no key, or a backup of your C:/Documents/Photos when you moved everything over to your D:/Documents/Photos, or C:/photos doesn't do you any good.

This reminds me of the user who thought he had Quickbooks (the only thing he cared about on this PC) backed up on a flash drive when his hard drive suddenly died. Sadly, all the flash drive contained was a copy of the desktop shortcut to QuickBooks.

i dont know of good way to validate that every single file is ok and that it copied without corrupted bits and pieces. Every time i create copy, i create full copy of the original, i dont do incremental backup. Any advice?

sirmium wrote in post #18032576
i dont know of good way to validate that every single file is ok and that it copied without corrupted bits and pieces. Every time i create copy, i create full copy of the original, i dont do incremental backup. Any advice?

Some programs have the option to validate/verify the backup. That will at least confirm that the backup archive isn't corrupted, but won't confirm that the backup copy matches the original. The more independent backups you have, the better the chance that at least one of them will be readable when the time comes. Beyond that, all you can really to is spot check restoring files. If you backup consists of simple copies of you files in another drive, there are file compare utilities available.

I do it manually.

I recently did a 50gig backup as I do it in spurts every few weeks. I manually copy what I want to keep copies of to two separate external hard disks via USB 3.0, which then go into separate boxes to keep from being banged, dropped, etc (but not fire proof). And a select few get uploaded into my cloud. Three separate copies of all my important data. I don't keep tons of junk, I try to cull as much as I can before doing this.

When ever I do my backups, I look at previous data randomly to make sure the medium is healthy. If I even suspect something, I replace the drive and use my other backups as sources if necessary.

I rotate out my drives about every 2 years for new ones (both for health of the drives and for increase in drive capacities with falling costs every few years making it very affordable).

Currently rotating 5 TB of actual data in various media forms.

Very best,

I cull a lot as well. Probably better to shoot less, but in the "how many photos do you shoot per wedding" discussion, Im in the middle of the pack

Its not much more work to delete at the very least all the obvious junkers and I see no reason to keep them around.

My inspection method is to skim through and open the files. The bulk of the data I'm backing up (images for the most part) are all very similar in file size, so that makes an easy way to spot critical backup errors. If all the files are 5kb rather than twenty some MB, then I know something obviously went wrong.

I also generate a hash manifest of the files in the archive. This file is easy to confirm as still valid as I can check that the xml tree is still intact, and then compare the redundant metadata. Would need to dig out some notes to confirm details, but the short of it is that each file gets an entry in the manifest as part of the automated archiving system:
- File name/location
- Hash of the file's data, excluding file metadata of things like "date last accessed" that can change, which is a short code of characters generated based on the data.
- Error detection strings of file name and hash value. Short mathematical code that can be compared.

The odds of a file error keeping the XML intact are terribly low in and of themselves. The odds of a file error also changing anything in the above values in a way that still line up and not raise a flag are insanely low.


However, the deep inspection of regenerating the hash values for everything in the archive does take time: Unzip to temp copy, run data tests, compare to manifest, does take time. Luckily it still runs through in less than 10 hours, so it mostly means leaving a computer to run over night a dozen times a year.

However, just loading up the data and looking through the photos is probably 'good enough' for the majority of people. Be as detailed as you feel you need to be for your data, but just make sure it is actually there. Far far too often I see stories about people who only ever reach for their backup files when there is a problem. And when you have a problem and you need your backups is not the time to discover that there was already a problem there as well.