Accuracy of Raw vs. JPEG in camera histograms.

We had a discussion recently about in-camera histograms. Magic Lantern has software that allows the evaluation of in-camera raw histograms as distinct from the histograms Canon software presents which are based on the processed jpeg determined by color balance, picture style parameters etc. They have an explanation and illustration towards the bottom of the post here

The issue was raised by observations that Canon's processing, to a jpeg for Live View, exaggerates the exposure in the histogram compared to what could be optimally recorded by the sensor.

In the bottom right hand corner of the ML shot is the raw histogram showing the red and green channels just to the left of the right edge - i.e. below clipping/saturation. (The D10.6 is a live estimate of the dynamic range for the shot)

Good info, thanks for sharing!

Magic Lantern released a version that they say will work on the 5D Classic, but I downloaded it and tried to install the firmware and got a FAIL, don't know if I flubbed it or what, I'll have to check with them!

tonylong wrote in post #16984739
Good info, thanks for sharing!

Magic Lantern released a version that they say will work on the 5D Classic, but I downloaded it and tried to install the firmware and got a FAIL, don't know if I flubbed it or what, I'll have to check with them!

Tony, that s/w is very flawed and no longer supported. Its caused my 5d to lock up quite a bit so I uninstalled it. Be careful of it.

PS check my posts on the ML forum

WaltA wrote in post #16985346
Tony, that s/w is very flawed and no longer supported. Its caused my 5d to lock up quite a bit so I uninstalled it. Be careful of it.

PS check my posts on the ML forum

Ah, OK, I'll stay away from it, thanks Walt!

The histogram on the camera's LCD is calculated using whatever white balance and/or picture style setting you have configured in the camera, essentially showing the graph applied to the in camera rendered JPEG. If you want the histogram to be more accurate, simply go into your picture style and lower the sliders for contrast and saturation all the way down. You really don't need any additional firmware or SW to get a more accurate idea of what the sensor has captured.

bsmotril wrote in post #16985633
The histogram on the camera's LCD is calculated using whatever white balance and/or picture style setting you have configured in the camera, essentially showing the graph applied to the in camera rendered JPEG. If you want the histogram to be more accurate, simply go into your picture style and lower the sliders for contrast and saturation all the way down. You really don't need any additional firmware or SW to get a more accurate idea of what the sensor has captured.

Good tip! Never thought of this even though I've been aware of the discrepancy for years.
Has anyone tested this to be sure? I think I'm going to set all my cameras to a custom style now.

Good thread

ML is no longer supported? Do tell.

The Canon histogram is correct for people who shoot JPEG only.

CyberDyneSystems wrote in post #16985641
Good tip! Never thought of this even though I've been aware of the discrepancy for years.
Has anyone tested this to be sure? I think I'm going to set all my cameras to a custom style now.

Good thread

That's why many people dial contrast down to a minimum - the highlights are not "multiplied" so much and don't get as close to the right hand side.
The processing for the preview is obviously necessary to see a reasonable image, but the histogram is often/usually what people go by to judge exposure settings.

If you don't want to process after capture, then you want the color channels not to be blown out in the processed image, so the presentation of the processed histogram is appropriate for that situation. In the above example, you would need to dial down the exposure to get the (processed) red values down to below saturation/clipping/bl​owout for the resulting jpeg. That means the actual captured values in the red channel are a long way below maximum (so they don't blow out after being multiplied as WB is applied) and results in some possibly significant loss of overall captured DR. Most of the time, that's not going to matter much, if at all, to the end result.

If you are prepared to process after the fact and therefore shoot raw, you may well be someone who wants to get the most DR at the capture stage. That means knowing that the unprocessed values for each sensel are below max and not blowing out (even if they might blow out when the data multipliers are executed for jpeg processing). So that's when you need to use the raw histogram as a judge of how far to the right the histogram should go. So the "best of both worlds" here is to have the preview based on typical processing preferences but use the raw to judge the actual exposure settings. Then you wouldn't have to have a washed out greenish preview just to get max DR. Then you can start PP on the raw data knowing that nothing was blown out (in the regions of interest) and that the shadows are as good as they can be for that image.

The histogram display in ML also has little indicators when any channel blows out and what % of pixels in that channel are doing it, so accommodating some % of blow out in e.g., specular highlights, is easier. Whether this approach is worth the effort is obviously an individual opinion. (Of course, if you also like a built-in intervalometer, unlimited HDR bracketing, automated focus stacking, focus peaking, trap focus, auto ETTR etc, then the "worth" of ML might increase - and then there's all the video stuff).

It seems it would be quite simple for Canon to have a CFn for the histogram display to switch between jpeg and raw for this purpose. If they "let" us shoot raw, we should have the best tools for assessing its capture

I've just tried taking some sample shots of a scene (same exposure settings, just keeping profile set at Faithful and going from a contrast +saturation of 0, -2, -4). When looking at the brightness histogram, I didn't see much of a change in the histogram but did see one in the preview jpeg.

In most instances, I have found I can gauge exposure based on the brightness histogram and faithful image profile. I've also found there's a feature in the 5DmkIII (and other recent Canons) that lets you preview and adjust the exposure range of the RAW (RAW image processing). It can save over the jpeg version, but looks like it's another tool where you can go up or down 1EV to check highlights and shadows (would be nicer if it went up or down at least 3EV):

https://www.youtube.co​m/watch?v=e-OUhzMuVBM

CyberDyneSystems wrote in post #16985641
Has anyone tested this to be sure?

Yes: Picture style Faithful, contrast to -4 and saturation to -3. (5D3)

I compared the blinking areas on the LCD to the overexposure warnings in RawDigger and they matched nicely. Can't be shure that it works so well on just every image though.


Edit: Shooting ETTR, I prefer blinkies to the histogram, for two reasons.

The line in the histogram, denoting clipping, is hard to see. Besides, I like to know where in the picture the clipping is taking place, as it may be acceptable at some locations.

The effectiveness of dialing down the Contrast all the way depends on the scene and how you approach exposing it -- Contrast affects the highlights and shadows, not so much the mid-tones, so unless you are pushing your highlights "To The Right" you may not in fact see a substantial difference.

If you want to "play" with the idea, take a target that is white but with real "detail", like a white paper with print, a white feather, whatever. Set your exposure to render white as white, but without clipping, in other words so that the details on the paper/feather are properly rendered. But do this initially without relying on the histogram. In other words, spot meter the white "target" and then set your exposure to what would be "pushing" (to the right) but with Raw should also retain some detail.

For testing purposes, I've seen a +3 EV exposure for white actually retain detail, although you'd need to crank on your Hightlight Recovery software (I hear that the new DPP4 has some real gains in the Highlights).

If you've satisfied at the level at which you can push your white exposure, then, take a look at what your in-camera histogram and "blinkies" tell you, and play a bit with especially the Contrast, moving it from zero back to -4 and compare. If you have Live View you can compare the histogram before a shot and after, see what difference it makes!

In practice, I don't tend to push things "all the way", although just the knowledge that I have a bit more headroom than even my histogram and blinkies suggest with the Contrast dialed down gives me a bit of reassurance. Some scenes have highlights that you can "give up", of course, but it is a real pleasure when you can recover more than you realize!

While not exactly a "groundswell", I am noticing more people becoming interested in a raw-based histogram. I think it would be really nice to have that option but I think I understand why camera manufacturers don't do it.

I think the reason is that many photographers don't have a good grasp of color management and going from wide-gamut to smaller-gamut color spaces has a bit of a learning curve to manage properly. You can see this lack of understanding still prevalent on photo forums with posts along the lines of "Canon is extra sensitive to reds", "the camera light sensors don't correctly measure highly-saturated colors", etc.. The truth is that probably a good 80% or more of "blown reds, oranges, and yellows" is due to trying to fit those colors into the sRGB color space. The raw file itself is perfectly fine.

agedbriar wrote in post #16985805
Yes: Picture style Faithful, contrast to -4 and saturation to -3. (5D3)

I compared the blinking areas on the LCD to the overexposure warnings in RawDigger and they matched nicely. Can't be shure that it works so well on just every image though.


Edit: Shooting ETTR, I prefer blinkies to the histogram, for two reasons.

The line in the histogram, denoting clipping, is hard to see. Besides, I like to know where in the picture the clipping is taking place, as it may be acceptable at some locations.

With those parameters, the number of cases where the histogram will mislead seems to be pretty small. The actual display of the Canon histogram has the problem you mention (it's been there as long as they've had histograms, almost as long as they've needed a MLU button, but we digress) so the right edge is hard to see. By the same token, with those parameters dialled in, the blinkies will be a pretty good reflection of the clipped areas so you can blow out only the areas that don't matter.

davesrose wrote in post #16985784
I've just tried taking some sample shots of a scene (same exposure settings, just keeping profile set at Faithful and going from a contrast +saturation of 0, -2, -4). When looking at the brightness histogram, I didn't see much of a change in the histogram but did see one in the preview jpeg.

In most instances, I have found I can gauge exposure based on the brightness histogram and faithful image profile. I've also found there's a feature in the 5DmkIII (and other recent Canons) that lets you preview and adjust the exposure range of the RAW (RAW image processing). It can save over the jpeg version, but looks like it's another tool where you can go up or down 1EV to check highlights and shadows (would be nicer if it went up or down at least 3EV):

https://www.youtube.co​m/watch?v=e-OUhzMuVBM

The 7D had this too, but not exposure, and even though the 5D3 has exposure, it is one stop either way, which seems odd to me.

Faithful is okay, neutral might be better, and even those have adjustments made for you under the covers that you cannot see. You might be able to see what those are with the picture style editor. Perhaps we could build our own picture style with Canon's editor and upload it back to the camera to be used when we want to see a truer rendition of the histograms, at least the brightness. The bigger issue is that the raw file has at least 1/2 stop extra headroom at each end, and there is no way to make the histogram in-camera show more range, correct?

http://cpn.canon-europe.com …features/pictur​e_style.do