EXIF Summary Tool for 500,000 photos?

I have shot over 500,000 photos so far, and Lightroom has this feature that allows me to see out of the 500k photos, what are the FLs/aperture/SS I used for my 5-star rated photos as a whole.

Is there any other tool out there that does the same? The reason I'm asking is because I simply need it to read and summarize the EXIF data, instead of wasting time building thumbnails etc.

Any advice will be appreciated!

In my other life (outside of photography), I'm a programmer/analyst. If you are comfortable using Python and related Unix/Linux utilities, there is a module called exif.py that supposedly will read exif data from both jpeg and tiff files. I've not used this, so I can't tell you if or how well it works. However, I've been writing Python code (version 2.x) for years, and it is simply excellent. My guess is that with a bit of scripting, you could easily process the exif data from 500,000 photos. (If you can process it from one, you can process it from half a million.)

If you use Linux, this should be very easy to do. If you have a Mac, it should also be very easy to do as its brain is Unix. Additionally, Python is installed in the Mac OS. If you have Windows, Python is available for you to install. While you might have a bit more of a challenge scripting the processing of 500,000 files, I've found Python to work well under Windows.

All that said, if you are not a programmer or have never used Python, this really isn't your option. Easy as Python is to learn and to use, there's still a considerable learning curve to it.

Hope this helps!

Unfortunately, I have no background in programming and it would be tough for me to pick it up just for this purpose.

Any other suggestions?

Thanks!

How about ExifTool? This is command line, but there is a ExifToolGUI option too, with a graphical interface. just google it to find it.

Alan

the.forumer wrote in post #18228178
I have shot over 500,000 photos so far, and Lightroom has this feature that allows me to see out of the 500k photos, what are the FLs/aperture/SS I used for my 5-star rated photos as a whole.

Is there any other tool out there that does the same? The reason I'm asking is because I simply need it to read and summarize the EXIF data, instead of wasting time building thumbnails etc.

Any advice will be appreciated!

Have not tried it but you can take a look at "Exposureplot."


http://www.vandel.nl/e​xposureplot.html

ExposurePlot is very close to what I want! The only issue is I can't seem to be able to filter by the star ratings pegged to my JPEG output files.

Would be most grateful if anyone knows how.

the.forumer wrote in post #18228734
ExposurePlot is very close to what I want! The only issue is I can't seem to be able to filter by the star ratings pegged to my JPEG output files.

Maybe take a look at the image viewer he has, Wega2 (same site) ... I think the exposureplot function is built-in to the image viewer, which may allow you to filter.

I have just checked with the developer, and they said there's no such function in Wega/ExposurePlot.

Still looking for suggestions!

Any Use?

So what exactly are you wanting from this data? Just to view the images? If so are they all RAWs? Are you using LR anyway? Extracting the EXIF data from 500K individual files is not an easy matter, since you literally have to open each file, read the required data, from however many of the fields you want, and by the time you add the maker fields that can be a lot. Then you have to store the extracted data somewhere while you read the next file, rinse and repeat 500K times. Then once you have extracted your dataset you then have to do whatever it is you want with the data, and at the moment I'm not really sure what that is going to be. The thing is that if you are using LR already for your image management you already have all of this data in a nice easy to access location, the Catalogue file, which is a standard SQlite data file. Even so you would still need to have the necessary skills to make use of it. There are I believe some add ons for LR that allow you to collate information pulled from the various fields of the LR database, so that you can perform a degree of analysis on your images. If you do use LR I would suggest looking at Jeffrey’s Lightroom Goodies (Plugins and Tools) he might have something of use to you.

Alan

the.forumer wrote in post #18229618
I have just checked with the developer, and they said there's no such function in Wega/ExposurePlot.

Still looking for suggestions!

I'd like to contact the developer ... please PM me his email or contact info.

It can be found on his website.

Any other suggestions?

If you are wanting to produce graphs to show the distribution of various camera settings across a set of images then you are going to need to do a bit of DIY to get exactly what you want. it's quite simple really, you need to have a batch file that will run EXIF Tool on all of your image files, and for each one extract all of the fields that you will be interested in. It will then need to append the data to a text file, with each images data on a new line. Ideally you also want to ensure that each item of data on each line is separated by a comma, and that any empty fields are included in this. You could use spaces to separate fields, but only if you know there won't be any spaces in the actual data, and also that all fields will contain valid data.

Once you have generated the text file the next step would be to load it into a spreadsheet program, which will then allow you to conditionally count the numbers of different variables in the various fields. This may require converting some of the values in fields from the text they are imported in as to actual numerical values. Most spread sheets will do this during the import process though, as long as they see just numbers in a field. Once you have counted the variables then you just have it build you distribution graphs. Yes it is a lot of work.

The problem is that it is only a very tiny minority of photographers that would be interested in something like this, so there is not much demand driving development of software. If the above is too difficult/daunting then you could just load up LR and it will quickly give you counts of the number of photos in the catalogue that meet whatever requirement you are looking for. LR building previews does take time, but you can still search the whole catalogue while it is doing so, and it will also only generate the previews when you look at the images themselves in the catalogue. The thing is that added to the small number of potential users for what you want is the fact that many of those are using LR anyway, If you need graphs and use LR then again it is often very easy to manually copy the data you need across to a spreadsheet. The full range of f stop values that would need copying across if you went from f/1 to f/22 in third stop increments is only 30. It's a bit harder for shutter speeds, for Canon DSLR's there are either 18 or 19 stops of shutter speed available, since some go to 1/4000s and some 1/8000s, although the majority of shots by most photographers are likely to fall well within a ten stop range. ISO is also pretty much a ten stop range too, since that gets you from ISO 100 to 51200, and a ten stop range at third stop increments is 30 entries. The real killer for manually transposing from LR is focal length, especially if you have a long telephoto zoom. Checking my use of 150-600mm lenses, both the Tamron I rented, and the Sigma C that I own shows that combined I have taken 6282 images that have made it into LR, and in the process used a total of 65 discreet focal lengths. If I look at all of the lenses that are included, where the focal length is known, then I have a total of 150 different FLs recorded in EXIF from 10mm out to 600mm. It took me about ten times longer just to write the numbers down in this post than it did to look them up in LR.

So because of all of this I really don't think you are easily going to find a simple canned solution to your needs.

Alan

Try this...

https://www.lightroomd​ashboard.com/

mclaren777 wrote in post #18238253
Try this...

https://www.lightroomd​ashboard.com/

It still has the problem that Exposure plot had for the OP, no way to directly filter for star ratings. Although it would be very easy to do a new catalogue export using just the images with the required star ratings. Looking at some of the comments on the site I do wonder at the size of my catalogue, which currently has a total of 41998 images in the All Photos view, as the size of my catalogue is 17271396 KB, yes that is over 16 GB. To have a look at my data I used the extraction tool to reduce the size of the catalogue, since over about 1 GB seems to be able to stall app. The reduced file, with just the used information was only 18 MB. I do make a LOT of use of the local brush, and usually have a great many small detailed brush strokes, which I know can make for a lot of text to save in describing them.

Alan