CR2 file problems

[mminnig]

OK... for two weeks I have been having intermittent problems with CR2 files that appear corrupt or will not decode to JPG format in Digital Photo Professional.

I have finally replicated the error in as small of a test scenario as possible. Any help or suggestions as to how I can pin down the root cause would be TREMENDOUSLY appreciated!

1. I shot 314 files in RAW format on a 2 month old Sandisk Ultra II card. The card was formatted in-camera before the shoot.
2. I copied the CR2 files to a new directory on a brand new and freshly formatted internal hard drive.
3. I opened DPP and instructed it to batch process all of the files to JPG format on the same drive in a new directory.
4. DPP gave me 6 "Decoding Failed" errors.
5. The CR2 files in question open with no errors in DPP as well as Photoshop with ONE exception.
6. The one exception will not open in either program and is described as being corrupt.
7. I had to re-copy the one exception from the memory card to the hard drive and then it worked perfectly.
8. A second attempt to individually batch process each of the CR2 files in question produces no errors.
9. HOWEVER. Many of the JPG files that appeared to process correctly actually are garbled and missing data when viewed.

Personally, the only conclusion that I can come to is that DPP is the cause of the problem. However, for all of 2008, we followed the same workflow and did not encounter ANY of these errors. They just started creeping up in the last two weeks. Before anyone asks, yes, I did uninstall and re-install DPP.

Help?

[tim]

This is often traced back to a faulty card reader, or sometimes a card. You can't say it's DPP until you try another RAW converter. Copy the same file to another computer using a different card reader and use different software to view it.

I have software to fully test CF cards, I don't have the link on my right now. PM me to remind me to post the link for you later if you want it.

[tim]

Here's the testing software (http://sourceforge.net/projects/usbmst/).

Give us more detail about how you're getting images off the cards. Camera and cable? Memory card reader? What software are you using to download the images?

[mminnig]

Ok.... I think I have really narrowed it down now.

I followed the same procedure but on a different PC with a different card reader. Same software, same OS. The results were flawless. No errors. No problems at all.

So, just to be definitive, I am copying the files from the card to the original PC over my network. I will then run the same batch process in DPP.

Hopefully there will be no errors and I think I can definitively say that the problem is the card reader!

To answer your questions.... on the original PC with the errors, I have been using a Sandisk USB 2.0 CF card reader. I simply use Windows Explorer to copy>paste the CR2 files. On the second, error free, PC I used the card reader built into the case.

[mminnig]

CRUD.

I copied the files from the second, error free, PC to the original PC via our network. I then ran a batch in DPP on the original PC and it produced 10 errors this time!

Why do the errors appear randomly? Why do they only appear on one of the PCs?

The only other test I can think to run is: I'm re-copying the files from the #2 PC to the one with the errors. Then, I'm going to run the batch over our network from the #2 PC.

Now I am thinking that it is DPP again. Unless there is something inherently wrong with the PC.

UGH.

[tim]

Could be bad ram. Run memtestx86 - google for it, it's free so don't get suckered into paying for it. It usually has to run for hours or even overnight.

[mminnig]

Hmmmm... now that you suggest the RAM....
I looked under the computer properties and it is supposed to have 4GB but only displays 3.17?

[tim]

That's normal, it's because you're running a 32 bit os. Just run memtestx86 overnight and see what it says.

[mminnig]

Thanks again. I figured it was a 32 bit thing but our other 32 bit pc actaully shows all 4GB. Is that just a difference in the BIOS version or something?

Anyway, running memtest now.

[tim]

It will vary from machine to machine, but in reality no 32 bit machine can address all 4GB. It's to do with memory mapped IO, the more ram the video card has the less normal ram the system can address. The reason is kinda complex, but google will tell you more.

Memtest runs in dos mode, fingers crossed it's this as it's an easy fix.

[mminnig]

Well..... I guess there are no easy fixes.
Memtest86 ran all night and completed 4 full passes with 0 errors.

[In2Photos]

Just a shot in the dark but are you running any anti virus software and is it different on both machines? Try running the batch with your AV disabled.

[mminnig]

I am running antivirus on both machines... I'll give it a shot with it disabled because I am totally baffled at this point.

1. I just copied the files over the network from the good PC to the bad PC. (eliminates card reader as a cause. the same CR2 files batched perfectly on the good PC.)
2. I ran the batch from the good PC over the network with the files reading from and writing to the bad PC. (eliminates DPP as a cause since it is using the install on the good PC)
3. The batch produced one error. "Decoding failed". Different file than any of the other times.

I can't figure out how this could be so completely random. I've eliminated every cause except the antivirus (ESET Nod32) so I guess I will give that a shot and let you know the results.

[In2Photos]

I am running antivirus on both machines... I'll give it a shot with it disabled because I am totally baffled at this point.

1. I just copied the files over the network from the good PC to the bad PC. (eliminates card reader as a cause. the same CR2 files batched perfectly on the good PC.)
2. I ran the batch from the good PC over the network with the files reading from and writing to the bad PC. (eliminates DPP as a cause since it is using the install on the good PC)
3. The batch produced one error. "Decoding failed". Different file than any of the other times.

I can't figure out how this could be so completely random. I've eliminated every cause except the antivirus (ESET Nod32) so I guess I will give that a shot and let you know the results.
If the AV isn't doing anything (and it may not) then I think you have to start considering either some hardware issues (run some stability tests other than memtest) or a corrupt OS perhaps.

[mminnig]

what stability tests should I run? I have until the end of next week (30 day window) to return this new PC to Dell.

[mminnig]

Ok.... Ummmmm.... I have no idea what to think now.

1. I turned off the AV on the bad PC. Ran the batch. It performed flawlessly.
2. I turned the AV back on and ran the batch again. It performed flawlessly.
3. Deleted the files.
4. Copied them from the good PC to the bad PC again.
5. Ran the batch with AV on again.
6. 5 errors and counting....
7. Deleted the files.
8. Copied them from the good PC to the bad PC again.
9. Ran the batch with AV off again.
10. 1 error in the entire batch.

WTF?!?! (sorry. this is irritating in the least)

[mminnig]

I have run the batch process 3 more times on the problem PC. Twice from DPP and once through Adobe Bridge. All have run without errors.

Am I wrong to be concerned? Random errors REALLY scare me when it comes to my client's files.

[In2Photos]

I have run the batch process 3 more times on the problem PC. Twice from DPP and once through Adobe Bridge. All have run without errors.

Am I wrong to be concerned? Random errors REALLY scare me when it comes to my client's files.
Only you can answer that at this point, but I think if I were having random errors ALL the time that would be a problem. If it only occurred every once in a while I would likely live with it. I assume you backup your files before deleting them from the card. In other words you have multiple copies. If not I would strongly urge you to change your habits.

Does DPP tell you which files had errors? Have you tried to rerun a small batch of files just from the error list immediately after they occur?

[tim]

LinX will stress your machine and report any errors. Let it runs for at least a few hours.

Also try uninstalling and reinstalling DPP. Worst case try to reinstall windows, which should be easy if it's a new PC and doesn't have anything on it yet.

[René Damkot]

Maybe a HDD problem?

[mminnig]

I think I need a vacation. Ok. Here is the latest.

1. I spent an hour on the phone with a Dell rep who connected to my PC. Assumed the problem was caused by a conflict with a current process and subsequently disabled everything... Of course the problem went away because it is so entirely random. Of course, two hours after we hung up, I found a "corrupt" file again despite having all of the processes turned off.

2. I visited a client today and shot 50 test shots in a church where we have a wedding to cover tomorrow. I came home. Popped the CF card in our good PC and copied the CR2 files to the shared hard drive on the bad PC.

3. I opened DPP and started viewing the files only to discover 4-5 of them are a garbled mess. Since they were only test shots I had already deleted them from the memory card so I couldn't check them on the card.

4. I ran Tim's USB tester software and it found errors on the card. Microsoft's check disk utility seems to have corrected the errors.

5. I fired off 50 more shots here in our office on the same card and put the card back into our good PC. Using DPP I checked them on the card and they were flawless. I then copied them over the network to the shared hard dive on the bad PC... 2/3 of the way through the copy I received an error saying something like "there were not enough system resources to complete the action". Clicking "Try Again" seemed to work. However, I found 2 corrupt files when I attempted to view them over the network in DPP.

6. So.... I put the CF card into the card reader on the bad PC and view the files from the card in DPP.... flawless. Then, I copy the files onto the hard dive in the bad PC from the USB card reader. Open DPP to view them and..... flawless. No errors.

Conclusion....? Our good PC is a Vista Home Premium system with 4GB of ram and is only 6mo old. The "bad" PC is a Vista Business system with 4GB of ram and is 2 weeks old. I have my doubts on "Insufficient system resources"...

All of these problems seemed to crop up when we got the new machine so my mind keeps going back to it. I'm about to return it to Dell since I'm still in my 30 day no hassle window.

[MaxxuM]

I think I need a vacation. Ok. Here is the latest.

1. I spent an hour on the phone with a Dell rep who connected to my PC. Assumed the problem was caused by a conflict with a current process and subsequently disabled everything... Of course the problem went away because it is so entirely random. Of course, two hours after we hung up, I found a "corrupt" file again despite having all of the processes turned off.

2. I visited a client today and shot 50 test shots in a church where we have a wedding to cover tomorrow. I came home. Popped the CF card in our good PC and copied the CR2 files to the shared hard drive on the bad PC.

3. I opened DPP and started viewing the files only to discover 4-5 of them are a garbled mess. Since they were only test shots I had already deleted them from the memory card so I couldn't check them on the card.

4. I ran Tim's USB tester software and it found errors on the card. Microsoft's check disk utility seems to have corrected the errors.

5. I fired off 50 more shots here in our office on the same card and put the card back into our good PC. Using DPP I checked them on the card and they were flawless. I then copied them over the network to the shared hard dive on the bad PC... 2/3 of the way through the copy I received an error saying something like "there were not enough system resources to complete the action". Clicking "Try Again" seemed to work. However, I found 2 corrupt files when I attempted to view them over the network in DPP.

6. So.... I put the CF card into the card reader on the bad PC and view the files from the card in DPP.... flawless. Then, I copy the files onto the hard dive in the bad PC from the USB card reader. Open DPP to view them and..... flawless. No errors.

Conclusion....? Our good PC is a Vista Home Premium system with 4GB of ram and is only 6mo old. The "bad" PC is a Vista Business system with 4GB of ram and is 2 weeks old. I have my doubts on "Insufficient system resources"...

All of these problems seemed to crop up when we got the new machine so my mind keeps going back to it. I'm about to return it to Dell since I'm still in my 30 day no hassle window.

So, all these issues are only on the 'bad' PC? Easy enough, go replace the computer ;) Seriously, you have what is called an I/O problem and worse, it's intermittent. If the Camera and memory card work find on other computers without a hitch 100% of the time then it's the bad PC. It could be bad power (PSU), memory, CPU, motherboard or even a shorting peripheral. You'd have to have all these parts to switch them all out to find the problem. Just replace it if you cannot diagnose the problem.

[mminnig]

I can't think of anything else to do other than send this &*%$ thing back to Dell...
Now I have to pull our the two internals that I installed and deal with returning it and waiting for a new one... UGH.

With intermittent I/O errors, should I be concerned with other files that resided on internal or external hard drives connected to this PC? Is there a chance that it would have randomly corrupted some file that I haven't opened in four months?

[MaxxuM]

I can't think of anything else to do other than send this &*%$ thing back to Dell...
Now I have to pull our the two internals that I installed and deal with returning it and waiting for a new one... UGH.

With intermittent I/O errors, should I be concerned with other files that resided on internal or external hard drives connected to this PC? Is there a chance that it would have randomly corrupted some file that I haven't opened in four months?

It depends on what part of the bus that has the issues. There is a quick way to tell if your HDD/CDROM controller is working. Just put a CD in your player with plenty of data on it (doesn't matter what) and get WinRAR and make an empty RAR file on your desktop then drag many files from the CD to the empty rar file. If you can then extract the RAR to a second drive and have no CRC errors then you're probably OK.

Have you opened up the computer and push all the connectors, cards and memory to make sure they are all seated properly?

[mminnig]

Unfortunately you went over my head with the RAR part...

I had the case open to put in the two internal drives (yes the problem was occurring before I did that) but I didn't think to check all the connectors. Could it honestly be that easy?

[MaxxuM]

Unfortunately you went over my head with the RAR part...

I had the case open to put in the two internal drives (yes the problem was occurring before I did that) but I didn't think to check all the connectors. Could it honestly be that easy?

Yep, it could be. WinRAR can be found HERE (http://www.rarlab.com/). After installing it you just right click the desktop then click NEW > WIN RAR and an empty RAR file will be created. Then you pop in a CD (data or music is fine) then drag all the files onto the RAR file and it will open up and you release the files. It will start compressing it. Then, when that is done select all the files and uncompress (Drag) them onto another drive. If there are no errors then your data should be fine (on the hard drives at least). As long as that data didn't come through the USB controller.

I would check all the connections first though. Turn off the PC, touch something metal then push each of the wires (all of them) onto their connectors. The memory needs to be removed then put back (reseated). You can look at this video to make sure you do it right at YouTube (http://www.youtube.com/watch?v=vLPuhVR1X5E&feature=related).

[tim]

4. I ran Tim's USB tester software and it found errors on the card. Microsoft's check disk utility seems to have corrected the errors.

If that USB tester program found errors on the card throw it away. That's the problem. You can't fix bad cards. Even if it's not bad throw it away, it's not worth the risk.

[mminnig]

Well since you have all been helping me through this, I thought I would give you what is hopefully the final chapter in this saga....

I called Dell this morning and got an RMA for the computer. I pulled out the two internal hard drives that I had in it and uninstalled all of my software before dropping it off for UPS this afternoon. Dell was actually great to deal with and is refunding everything AND paid for the return shipping.

At the same time I ordered a replacement system that should be here by the end of this week. Keeping my fingers crossed.

[Robf]

Well since you have all been helping me through this, I thought I would give you what is hopefully the final chapter in this saga....

I called Dell this morning and got an RMA for the computer. I pulled out the two internal hard drives that I had in it and uninstalled all of my software before dropping it off for UPS this afternoon. Dell was actually great to deal with and is refunding everything AND paid for the return shipping.

At the same time I ordered a replacement system that should be here by the end of this week. Keeping my fingers crossed.

Did they get you to run diagnostics?

I've had 1 dicky dell, out of about 20 we've bought. Similar thing, random corruption. Turned out to be a bad stick of ram. We had another go south on us after a power cut and spike... same thing, ram failed. But the diags picked it up both times, and the engineer was out the next day to fix it.

[mminnig]

No diagnostics... Just took it back. I was at the point where I had wasted entirely too much time on it and just wanted something to work out of the box. I would believe it could be the RAM as well but the memtest program that Tim suggested found nothing.