JPG & RAW - How to remove the Dupes?

I was doing some spring cleaning of my hard disks and noticed that I have a large number of duplicate photos as I used to shoot in both RAW and JPG.

Are there any applications that can be run which will help identify and remove the dupes?

I can't be the first person in the world who has this problem, so I thought I would post it up here to see if you guys had the solution.

Thanks for any advise!
Andy

Are the RAW and JPEG files in the same folders?

What operating system are you using?

Yes, the files are in the same folders. I'm using Windows 7.

You may not need specialized software; you may be able to use Canon's DPP or ZoomBrowser EX to identify and remove the Jpegs.

Do the JPEGs have the same file names as the RAW with the exception of the file name extension? (jpg vs CR2).

Andy_Cam wrote in post #9988643
Yes, the files are in the same folders. I'm using Windows 7.

Yes, file names remain as they were from the camera. It is just the extension that varies.

I have not seen the option in Zoombrowser but will go and have a dig now. Thanks!

At a command prompt for the top level folder, (for example) C:\Users\Andy\Pics\> del *.jpg /s

That will delete everything with a .jpg extension in all subfolders, so be careful!

You can locate your files quite easily doing a wildcard search.

For example, if your files are typically named IMG2018.CR2 and IMG2018.jpg you can find them by navigating to the root directory of your hard drive and typing in the search box: img?.*

That will give you all the files that share the common "IMG" in the file name; then you can filter by filetype to separate the CR2 from the JPEG.

To me that would be a safer way to work than deleting all your JPEGs without at least some visual verification.


I don't use ZoomBrowser but I know DPP allows you to view CR2+JPEG as either a matched pair or individually. In DPP, go to Tools > Preferences > View Settings and there's an option to either view CR2 and JPEG as a single file or individually. If you view them individually you will be able to filter the Raws from the Jpegs.

I just took another look at DPP and it has a command that will select all the JPEGs in a folder: CTRL+ALT+J
Not an efficient way of working if you have tons of photos but at least you'll know you aren't deleting any JPEGs you might need in the future.

Andy_Cam wrote in post #9988828
Yes, file names remain as they were from the camera. It is just the extension that varies.

I have not seen the option in Zoombrowser but will go and have a dig now. Thanks!

Dean Rachwitz wrote in post #9988850
At a command prompt for the top level folder, (for example) C:\Users\Andy\Pics\> del *.jpg /s

That will delete everything with a .jpg extension in all subfolders, so be careful!

Another thing you could try would be to just search the folders for jpeg file extensions and then delete.

Would take longer then Rachwitz's way but if your afraid of deleting something by accident (To the OP) then at least you get to preview them as thumbnails or whatever you choose before the delete

All good suggestions so far, but not quite what I was after. Unfortunately, before I realised the error of my ways, I had a fairly good mix of days with sometimes only JPG, sometimes RAW, sometimes both.

I estimate that there are around 5000 dupe photos, and really don't fancy going through the folders and manually deleting them. It looks like it may be the only safe option however!

Cheers,
Andy

Actually if you have it like that and there are programs that allow you to scan your system for duplicate file names, sizes etc..

I dont many off hand and I got to head off to work at the moment. I know the older version of Advance System Optomizer could do it nicely. Since then they changed the software to be rather clunky and not very friendly in my eyes. Most of these software have a trial or you can try to google for a 'duplicate file scanner' or something similar and possibly find one that is freeware so no buying or trial is needed.

This only works if the jpeg and raw are the same name though. If each folder starts as a new numerical sequence you would have to scan each folder individually instead of all of them at once.

Just to drag this back from the depths again. I stumbled across some software that does this recently and thought I would add the solution in case anybody else ever does a search for something like this,

DoubleKiller (free) will remove dupes to a point, but Doublekiller pro will allow you to set certain criteria to search on (date, a specific part of a filename etc) and has identified 17GB of dupes in my folders. The pro version costs a whole €15/ $20.

I wont post a link to the site but it comes up at the top in google if you do a search.

Hopefully this helps somebody at some point. It only took me 9 months to find a solution.

Windows 7 supports this function natively. In Windows Explorer, just go to the top level folder containing the JPGs of interest and enter *.jpg in the search box, then sit back and wait. The OS will search through that top level folder and all subfolders and give you a list of all matching files that are found. Assuming those are the ones that you want to get rid of, do Organize > Select All and then delete.

Rimmer wrote in post #11700222
Windows 7 supports this function natively. In Windows Explorer, just go to the top level folder containing the JPGs of interest and enter *.jpg in the search box, then sit back and wait. The OS will search through that top level folder and all subfolders and give you a list of all matching files that are found. Assuming those are the ones that you want to get rid of, do Organize > Select All and then delete.

The problem I see with a batch search for jpegs is that you will likely end up with jpegs you want to keep rather than just ones that are unneeded copies of Raw files. That could be nasty.

So, I'd say opening a folder in a Raw viewer (whether it process Raw or not), ensuring that no "keeper" jpegs are there by visually scanning the thumbnails, then sorting, selecting and deleting just the jpegs would be a "safe" way to go.