Histogram: is it good to follow or not

tzalman wrote in post #4488565
Bill - This is theoretically true, but in practice Canon, both in its Digic firmware and in DPP, does not preserve the RAW white point. In order to avoid the problem of false colors in the highlights it clips them at around 3600 (12 bit) which means that the RGB image has reduced DR. It's the greatest glaring fault in DPP.

I didn't know that. However if it depends on the RAW developer there is nothing to worry, just choose a tool that does not clip these highlights like DCRAW.
What I know is that the linear 12-bit range of the RAW files is not entirely used in the low end. In fact there is a black offset which varies from camera to camera where the RAW file contains no information at all and all RAW developers (some cameras do it before storing the RAW data) substract automatically.

When I have plotted undemosaiced 12-bit histograms of my RAW files I found no information below level 255 (may vary from shot to shot):

HTTP response: NOT FOUND | MIME changed to 'image/png'

_GUI_ wrote in post #4489598
I didn't know that.

Here is a quote from the review of the 5D by Phil Askey on DPR:

Dynamic Range
RAW headroom
.... Experience has told us that there is typically around 1 EV (one stop) of extra information available at the highlight end in RAW files and that a negative digital exposure compensation when converting such files can recover detail lost to over-exposure......
It would appear that the EOS 5D's RAW files contain just over one stop of extra highlight detail, although interestingly this was pretty much unavailable in any Canon RAW converter because of their conservative approach to potentially incorrect data.......
The issue with highlight data above the normal 'clip' point is that the RAW converter can not be sure that it is 100% color correct, it appears as though rather than risk false color the Canon converters simply discard it.
---------------

Because the DPP color rendition is very good, a trick I will sometimes use is to do two conversions, one in DPP and one in a converter that has good highlight recovery and then in HSV space to substitute the V channel from the second converter for DPP's V.

tzalman wrote in post #4488565
Bill - This is theoretically true, but in practice Canon, both in its Digic firmware and in DPP, does not preserve the RAW white point. In order to avoid the problem of false colors in the highlights it clips them at around 3600 (12 bit) which means that the RGB image has reduced DR. It's the greatest glaring fault in DPP.

I think that the reason this is done is because of hardware limitations and not some arbitrarily chosen limitation. The sensor response curve (signal out as a function of light in) becomes very nonlinear as it approaches saturation.

I think that the expression "RAW white point" is somewhat of a misnomer since RAW data is just that -- raw. The white point, black point, middle gray, and color compensations are all whatever you assign to them during post processing -- admittedly, DPP is severely lacking its ability to do much of anything much with the data.

bill boehme wrote in post #4490367
The sensor response curve (signal out as a function of light in) becomes very nonlinear as it approaches saturation.

I have to disagree a bit with this Bill. In real tests I have done with my camera, measuring RAW levels achieved vs exposure time, the response of the sensor looked really very linear in the R and G channels, and had a strange behaviour in the B channel but just in the last f-stop, due to a shift in the second last sample.

Think that HALF these linear ranges of the sensor ONLY represent 1 f-stop of the total DR. Looking at the R and G response, we can conclude they can be considered linear in the whole range.
And regarding the B channel, I think it was linear as well, simply the Time interval provided in the EXIFF for the second last sample didn't correspond exactly to the real exposure time.

HTTP response: NOT FOUND | MIME changed to 'image/png'

HTTP response: NOT FOUND | MIME changed to 'image/png'

HTTP response: NOT FOUND | MIME changed to 'image/png'

HTTP response: NOT FOUND | MIME changed to 'image/png'

That seems reasonable, but I think that I would need to convince myself that what you are measuring is a function of only the input variable and that exposure time changes have the same effect on the sensor as changes in light intensity if operating in a region of nonlinearity (assuming that it exists) by using a calibrated light meter (which I don't currently have). Eventually, the sensor will run into some sort of a non-linearity near saturation simply due to the nature of semiconductors operating in what is known as an open-loop condition (meaning that there is not some sort of feedback to linearize their output). However, the device may be designed such that it is outside the operating range of the portion of its output that is digitized.

bill boehme wrote in post #4494958
Eventually, the sensor will run into some sort of a non-linearity near saturation simply due to the nature of semiconductors operating in what is known as an open-loop condition.

Surely. In fact if you look closely to the histograms above, you can see some "distortion" in the shape of the blue channel histogram in the last f-stop (0EV) when compared to the others. The shape is slightly different to the others (it is a bit wider), but nothing very important actually.

Besides we must think that in that shot ALL R and G channels were blown, that means the sensor was working with 75% of the total pixels saturated and all blue pixels were surrounded by saturated pixels. I don't know the exact consequences of this regarding the electronic design of the sensor, but it seems reasonable to think that If under those extreme conditions the sensor was capable of yielding a resonable good quality in the B channel, we can consider non-linearities near saturation as negligible for regular photographic applications where the last f-stop will be by far less occupied than in this example.

HTTP response: 404 | MIME changed to 'text/html' | Byte size: ZERO

bill boehme wrote in post #4484393
Robert,

Part of this post is in response to your quoted text above, but the largest part of my post is directed to the subject, in general, so please do not feel that my more elementary descriptions are addressing your post.

With regard to your quote above, there appears to be a misinterpretation of what this actually means. What you have said would, indeed, be the case if each incremental count in an 8-bit word was mapped directly to incremental counts in a 12-bit word. In that situation, the four additional higher-order bits amount to an expansion of the range of values that can be expressed (or, as you state, four additional f-stops). However, doing things that way would not increase the resolution of detail in an image, but rather only extend the range of exposure.

In actuality, however, the range of a 12-bit (or 14-bit) image is SCALED to match the endpoints. This means that 0H is the same in all cases and represents zero brightness of that particular color channel. The same thing applies at the upper end where (for an 8-bit word vs. a 12-bit word), FFH and FFFH both represent the same maximum brightness value of that particular color channel. Doing things this way results in the same total dynamic range dynamic range in brightness, but now, with the upper and lower boundaries defined to be the same in both cases, the increments of brightness of a 12-bit color are scaled so that 16 increments are available for each single increment of an 8-bit color. An example of where a difference between 8 and 12-bit color resolution would be noticeable is a wide view of a blue sky where the color change is very gradual. The 8-bit image would show discernible steps in color change (called posterization) whereas the 12-bit image would have 16 intermediate color changes for each color change of the 8-bit image. The improvement in fineness of resolution normally means that the 12-bit image would not show any posterization (how well your monitor and video adapter handle the translation from 12-bits to its display capability is a completely different issue since it does not impact the image itself).

There is nothing, in particular, that requires 8-bit per channel digital images to equate to any particular dynamic range of light since we are talking about light on one hand and binary numbers on the other. However, the one thing that is convenient in using binary numbers to represent light is that that one exposure value change in light (one f-stop) changes the amount of light by a factor of two (either doubling or halving) and changing the most significant bit (MSB) of the binary word changes its value by a factor of two, also. Therefore, we can create a scale where FF determines a light level that is close the upper limit of the sensor and each f-stop decrease in light from there, also results in a decrease in the binary number by one MSB from the previous value. This is reason that the upper-half of the number of bits in a digital image file occupies the brightness level of the highest exposure zone and each subsequent f-stop decrease results in half of the remaining bits being within that exposure zone. It is easy to see that after about 6 f-stops, there are very few remaining bits in the LSB’s of the binary word. That’s fine because the lower practical limit of the sensor has been reached and any exposure zone that has only a handful of bits to play with can’t really show any useful detail because any light variation gets grouped into a few discrete values.

...

Dear Bill,
I've worried about some of these points for a few days now and have now convinced myself that I knew better.
However, I fell into a trap that is easy to fall into.

So, I think I now agree with most of what you say above.

Robert_Lay wrote in post #4500765
.... So, I think I now agree with most of what you say above.

Dear Robert,

I am glad to see that you do not necessarily agree with all of the things that I said because I frequently find that what I have said or written isn't exactly without error. This is especially the case sometimes when my thoughts are racing ahead a few paragraphs beyond what my slow fingers are typing and complete trains of thought get left out and then sometime later when I read what I had typed, I wonder about my clarity of thought at the time.

Or, as a vain friend once said, "I thought that I was wrong once, but then I found that I was mistaken about being wrong".

bill boehme wrote in post #4512955
Dear Robert,

I am glad to see that you do not necessarily agree with all of the things that I said because I frequently find that what I have said or written isn't exactly without error. This is especially the case sometimes when my thoughts are racing ahead a few paragraphs beyond what my slow fingers are typing and complete trains of thought get left out and then sometime later when I read what I had typed, I wonder about my clarity of thought at the time.

Or, as a vain friend once said, "I thought that I was wrong once, but then I found that I was mistaken about being wrong".

That is what is so neat about POTN - no matter how much we learn every day, we can still keep learning.