Storage...

[jameelm]

I want to be a professional photographer. In that respect, I believe in buying equipment not just to satisfy the need persay; I believe that if you buy the right thing the first time, the value and the quality will pay for itself. For example, I'm not of the mind buying less than "L" glass for price's sake on the premise that I can touch up the resulting image in photoshop to something to an "L" standard. I believe shooting it right the first time makes the difference. But this question is about storage.

I've been looking into storage options for the purposes of general operation and backup. I'm convinced the most equitable way consists of a balance between dvd burning for backups and hard disk storage. I've been looking at external hard drives in the 300GB range. My question is whether it's believed that I should be looking higher (e.g. 500GB) or if 300GB will suffice given the very low volume that I deal with now? That is, should I buy something I can grow into, or should I buy to "not have to think about it for a good while"?

Any and all feedback is welcome and appreciated.


jameel

[sageone]

I say if you're going to go after this thing hard core, go all the way. Get a 500 or a terabyte external...it'll cost a good chunk of change, but if you see yourself going trigger happy and you want to keep all your stuff, you'd better have a ton of storage.

[Wilt]

DVD that you burn yourself is questionable about long term storage life. If the purpose of DVD archive is to permit you to rebuild your data Saturday if your harddrive crashes and you buy and replacement harddrive at the store on Friday, it's fine! But if DVD is meant to be your long term access in 2016 to photos shot 10 years earlier in 2006, look elsewhere!
If you are wise, your on-line data is stored on a RAID, with multiple harddrives in the RAID which have automatic data redundancy. So if you write data at 10:21am and have a harddrive failure at 10:22am, you still can retrieve the data from the redundant harddrive at 10:23am! And then you also have daily backup to DVD which is stored offsite (at home, at your mom's home, in the Greyhound station storage locker!) so that if the whole RAID should fail (it DOES and HAS happened to people!!!) you can recreate you RAID with only minimal data loss!

[snibbetsj]

Might also think about tape backup. Also, it's a must to have one copy of everything stored off-site, preferably in a secure area.

[Phil V]

The question was about external HDD's so:
Will a 300gb unit fulfill your expected needs for the next 2 years easily?
If yes - buy it, because in 2 years time a 600gb unit will be less money than the 300gb today.
If NO - then buy something that'll last at least that long.

This isn't an L lens, you're not talking about buying better quality, you're buying pure IT, as such Moores law is the single largest factor.

Also bear in mind what everyone else said about backups, tapes, DVD's etc.

[KennyG]

There is another way to look at this. Your drive is a mechanical device prone to failure and it will probably do so at the worst possible time. 300GB is less of a loss than 600GB. You would be better off with mulitple smaller drives or preferably a RAID array, again of smaller drives.

The first question to ask yourself is how valuable is your work, not what is the cheapest storage solution.

[chtgrubbs]

I use a pair of 250GB drives in my computer in a RAID 1 for image storage of recent files, plus external 250GB discs for long term storage of inactive files, plus burn the RAW files to DVD.

[SWPhotoImaging]

I agree with Kenny's perspective on drive sizes vs. the failure factor. Last year I invested in an external, firewire 1TB RAID array for my main photoshop PC. It is made (or sold) by Micronet, and it was about $1500.
See an example here:
http://www.superwarehouse.com/MicroNet_1.0_TB_Platinum_RAID,_Firewire/PR1000FW/p/482921
This way, a single drive failure doesn't cause any loss of data. The entire array can be unplugged and carried away in seconds (this is both good and bad).
Given that I upgrade my storage hardware every couple of years, I'll be copying the contents from this array onto some replacement in a couple of years, thereby creating "fresh" files on new media. I may keep this as a backup if it is big enough . . .

[INNflight]

Basically, everything important has mentioned in this thread.

I can say I do support the idea of having several smaller HD's instead of fewer big ones, because of the above mentioned reasons.

If it comes to file storage, don't be shy to spend money. If you're doing this for a living, you wouldn't want to loose all your work because you saved a couple of hundred dollars by not buying a 2nd backup device!

Personally, I keep all files on an internal HD (photography files only on there) for an entire year.
The very same files are on two external 500GB HD's from the very first moment I download them from the camera.

Again, these very same HD's have two *brothers* ( ;) ) at my parent's flat, with the same files stored on them, updated once a month.

It would take a good load of s*** to loose them all at one time. :)

[Mike Panic]

since i work in the IT industry (at a pro camera lab) i'll chime in with my 2 cents about archiving digital images.

first and foremost... having at least 2 copies is essential. what no one has brought up yet is that they need to be at two different locations. if your house burns down and your dvd backups are in the filing cabinet under your computer table, where your computer is parked... then its pointless. you need to have them in two locations... work and home, home and parents, home and the bank vault, whatever.

next. cd or dvd backup is a standard... but you need to burn at the slowest possible speed and do a data verification afterwards. you should also check yourself. then place the disk into either a hard sided jewel case or into an archival grade, acid free album designed for such a thing and write on them w/ an acid free marker, not a sharpie, whic h will eat thru the label over time.

for your external storage needs. don't bother w/ a usb / firewire hard drive. for around a grand you can get a 1u or tower server graide piece from dell that has a hardware raid 5 in it, anything else is pointless. if your going to use spinning media, might as well make it fast and redundant at the same time.

be aware that NAS (network attached storage) devices are on the cheaper side of things, but if the installed version of windows dies, so does all the data on your drives, regardless of the raid, its gone.

if you are truely a beliver in doing it right the first time, do it right w/ storage, you can get 500-1000gb of raid 5 storage for under $1500 and feel good about it. they are easy to manage and in the long run, very worth it.

maintance. i would routinely (ever 3 / 6/ 12 months, whatever you want) check the cd/ dvd's you have burned for consistency. for what blank dvd's cost, it might b worth it to burn 2 at once.

also - be ready for change. the cd-r has barely been on the market 7 years now. i still remember paying $65 for a 50 spindle of cd-r's and thinking that was a great deal. now you can get a 100 dvd-r's for around $35. be aware that things are going to change and evolve. when they do, you need to get ready to move your old disk storage to the new system. blu-ray is around the corner and so is hd-dvd-r's... its hard to buy a 3.5" internal floppy drive anymore, much less floppy disks. same goes w/ zip drives and disks... so you want to avoid legacy items and upgrade as soon as you can finacially do it.

[Wilt]

If the external HDD is USB, one could power it up ONLY when writing/reading data with that drive. That prolongs even the statistical life (the MTBF figure) since it is not accumulating hours while in the powered-down state. Powered up drives start the inevitable countdown of the MTBF count, and it has been stated by some manufacturers about 'designed to last 3 years' if left powered on, in spite of the statistical hours that the MTBF figure misleads most to assume.

--wilt

[Roach711]

There are lots of ways to lose your picture files. You may delete them, your hard drive may fail, the CD/DVD you copied them to may become unreadable, the computer may be stolen or the house may burn down. A well thought out backup strategy will:

Ø Protect your pictures against a variety of calamities
Ø Be convenient and easy to do
Ø Make your files easy to find

My Backup Workflow


My PC is set up with a main hard drive (HD) which holds program files and data, and a second physical HD which is solely for backing up files. If you use this setup make sure this second HD is a physical HD not just a second partition on your main HD. A second partition on a primary HD will die along with the primary drive where a second physical HD (almost) never does. Instead of a second internal HD you could use one of the new inexpensive external USB hard drives. These external drives can also double as your off-site storage media.

Here’s my workflow:

1) On my PCs main hard drive (HD) I have a folder named DATA. All of my documents, MP3 songs, databases and pictures are stored here making backups a one-stop deal. In my DATA\PHOTOS folder I create a new folder with a name that’s descriptive. For example, “2006-5-22 Fluffy’s Birthday Party.” This quickly tells me the date, sorted by year and month, and a description of the pictures.
2) Next I copy my files from the camera’s CF card to the new folder on the PC using a USB 2.0 card reader. The card reader is much faster than attaching the camera to the PC directly. I leave the files on the CF card for now.
3) Now I copy this new folder to my second HD so I have redundant copies of the files.
4) Using Adobe’s DNG converter I convert my RAW files in the original folder to the DNG format and delete the original RAW files. The .dng files are nearly 1 meg smaller than my RAW files saving a significant amount of storage and backup space. The original RAW files are on the backup drive and can be accessed if needed.
5) Using Adobe Bridge I go through the picture files in the original folder deleting the obvious dogs and those that are technically OK but would never be printed. I’m getting more and more aggressive with my deletions as I get older and may end up with 10% (or none, if it was a bad photo day) of the original photos when I’m done.
6) Once I’m happy with my deletions and post processing in the original folder I copy the entire folder to the second HD replacing the folder previously copied here. I’m now protected against accidental deletions and primary HD failure but could still lose files if the PC were stolen or the house burns up.
7) I can now format the CF card in the camera if I need the space.
8 - Whenever I have enough new files that I become a bit paranoid I copy the entire DATA folder to the second HD and do a full backup to DVD using a program called Archive Creator. This program lets me copy all my files to DVD and spans DVDs so I don’t have to decide how many files will fit on each DVD, which can be a major pain. The program figures out how many files to burn to each DVD and all I have to do is feed blank discs into the drive. It also verifies each disk as it is created. If I don’t have enough new files to warrant a new full backup but still want an interim backup I’ll copy the new files to a re-writable CD and take the CD to work for temporary storage. Once I do a full backup the re-writable CD goes back to the used disk pile and gets erased only when I need the disk for another backup.
9) This new DVD backup set goes to work with me and is stored in a cool, dark place. I leave as many backup sets offsite as I have storage space for and bring those that won’t fit home and archive them, also in a cool, dark place. This gives me multiple generations of backups. If one fails another likely will not.

I’ve already been saved by this workflow when my primary HD went south. I was able to copy all my files back from the backup HD. In the event of a stolen PC or a house fire I can bop over to the office and grab one or more of the stored backup sets. When DVDs become old technology I will simply begin doing full backups to the new media with no files being “orphaned” on the old media. In the event that I acquire so many files that full backups become unwieldy I’ll archive some of the older files to another folder, which I’ll also copy several times to backup media and store off site.

This approach has flaws, of course. My full backup sets will eventually become very large and will take more and more time to burn. At present I’m using 6 DVDs per backup. Hopefully, keeping only the best shots will help alleviate this and new drives with higher-capacity backup media will also help.

I store backup disks in a cool, dry place as heat and humidity can cause the dyes in writable media to degrade. Finally, I’m careful when removing CD/DVD disks from their jewel cases. Bending them too far can break them. DVDs seem to be more fragile than CDs. Pressing on the case’s raised plastic hub releases the disk easily and avoids damage. Re-writable media is considered to be less reliable than writable media so avoid using CD/DVD-RW disks for long term storage.

Basically, do all you can to avoid a single point of failure. As our friend Murphy said, “If it can happen it will happen.”