Overexpose to reduce noise

chauncey wrote in post #12970353
On what are you basing the overexposure information...if it's the LCD histogram, that can give you only a ball park semblance of accuracy as it's based on a jpeg rendition of the image which, to a large part, is controlled by the picture style settings that have been inputted.
Your RAW converter, whatever it may be, is a much better indicator of exposure.

I'm basing the exposure on an incident meter reading using a meter that's calibrated to the camera body.

I've had all my bodies set to neutral picture style with zero sharpening and -4 contrast for a few years now to get an as unbiased a histogram as possible. But this is not playing into the test at hand.

I'm finishing this experiment, so here are the last images:

testing to see how the images would compare shooting at ISO400 and ISO800, same aperture same shutter speed. The ISO400 image would be EV -1/3 stop and pushed up in post, the ISO800 shot would be EV +2/3 and pulled back in post. The last image is ISO1600 at EV +2/3 pulled back in post.

It doesn't show up in the web jpeg images, but the ISO800 that is EV+2/3 has less grain than the ISO400 EV-1/3

40D
35L
F2.8
SS 1/20

1)ISO400 EV -1/3
2)ISO800 EV +2

HOSTED PHOTO
please log in to view hosted photos in full size.

HOSTED PHOTO
please log in to view hosted photos in full size.

ISO1600 EV+40
SS 1/40

HOSTED PHOTO
please log in to view hosted photos in full size.

windpig wrote in post #12971972
I'm finishing this experiment, so here are the last images:

testing to see how the images would compare shooting at ISO400 and ISO800, same aperture same shutter speed. The ISO400 image would be EV -1/3 stop and pushed up in post, the ISO800 shot would be EV +2/3 and pulled back in post. The last image is ISO1600 at EV +2/3 pulled back in post.

It doesn't show up in the web jpeg images, but the ISO800 that is EV+2/3 has less grain than the ISO400 EV-1/3

40D
35L
F2.8
SS 1/20

1)ISO400 EV -1/3
2)ISO800 EV +2

The idea is to NOT underexpose. ISO 400 -1/3 EV would be underexposing if you need to push it by 1/3 EV to get the exposure right.

PacAce wrote in post #12972138
The idea is to NOT underexpose. ISO 400 -1/3 EV would be underexposing if you need to push it by 1/3 EV to get the exposure right.

Leo,
go play with your grand kids or something:p

Read my lips, "no new taxes".....I mean, the idea as I massage the s&$t out of this experiment, was to see if there was an advantage, with compositions that had no highlights to protect, to overexposing and pulling back in post if one needed a slightly faster shutter speed (nice run-on sentence!).

As a base line, I used the same shutter speed of 1/20 and the same aperture, one would then have to under-expose the ISO400 shot by -1/3 stop and over expose the ISO800 by 2/3 stop get the same exposure per the indecent meter.

The ISO400 shot would have required a SS of 1/15 to get the correct exposure without a PP adjustment.

So here are two images to compare:

No NR
ISO 400 at zero EV, SS 1/15
ISO 800 at EV +2/3 pulled back in PP, SS 1/20

I think we need to do some more hair splitting, but it looks like it would be better to shoot at the lower ISO with zero EV if you can get away with it, but better to shoot at the higher ISO and EV +2/3 if you can and need the extra shutter speed.

I'll shut up now and go get some work done on the house

HOSTED PHOTO
please log in to view hosted photos in full size.

HOSTED PHOTO
please log in to view hosted photos in full size.

LOL, Ralph. Wasn't trying to give you a hard time (yeah, right!).

But I'm glad to see you undertaking these experiments, studying and interpreting the results and sharing it with the rest of the forum. Thanks for your thoughtfulness.

PacAce wrote in post #12970030
For me, the only time it would make sense to use ETTR is when the camera is already at ISO 100. At the higher ISO settings, if you can ETTR, then that mean you can also shoot at a lower ISO equal to however many stops (or fraction, thereof) of ETTR you are applying, and expose normally.

In most ISO ranges of most cameras, that is true. For all Canon DSLRs other than the original 1D, that is not ideal in the low ISO range, as the ugliest part of the noise is incurred after the analog "ISO gain". IOW, there is more ugly read noise if you expose "normally" as compared to doing the same sensor exposure at a higher ISO. As you go above 800 - 3200 or so, this difference becomes smaller and mostly disappears.

If you guys want to see something really scary, try a test that will really push things.

BTW, John, good to see you here and I know I'm not posting anything new to you.

Anyway, set up a scene where you can get a "good" exposure in ISO 1600, pushed to the right a bit if you wish but with no blown highlights and no blocked shadows. Shoot the scene in Raw and in manual mode and with any in-camera settings for noise, exposure corrections and such turned off. So, what you get is the aperture and shutter speed letting in light, and the light is modified by the ISO amplification, period. The only thing of difference other than the ISO that we are concerned about is the noise collected by the sensor and then that collected "upstream".

So, we will change the one variable we can. We will keep the aperture and shutter at the same physical settings, so the same light is being collected, but we will lower the ISO (the internal amplification of the signal) to the "cleanest" ISO that Canon cameras "boast" of, ISO 100.

Now, this is 4 stops lower in Ev, "horribly unexposed", but the only difference in our settings is the ISO internal amplification! As to light, the exposure is the same!

So go ahead, you are in Manual, just lower the ISO to 100 and shoot away! After all, we all learned back in the film days that a low ISO is "less grainy" and using a high ISO "causes grain/noise", right?

And then bring your results into your Raw processor of choice. Now, it has been said that some Raw processors don't handle such things as well as others, I don't know where the lines are drawn. Raw processors use their own internal processes to handle things.

But, the idea is you now have to boost your ISO 100 shot by 4 stops to be the "equivalent" of the ISO 1600 shot. You can do this in a straightforward way with the ACR/Lightroom Raw processor. In DPP you will need to set the Raw tab to Linear processing and then boost the Brightness to the max of +2 Ev and then go to the RGB tab and boost the brightness until the levels match the ISO 1600 shot.

And now you can compare "real" ISO with the "real" native 1600 ISO.

You may be surprised! There has been a lot of analysis/evaluation that points to the fact that the lower Canon ISOs are more noisy than the higher "native" ISOs! This does tend to level out at ISO 1600, maybe more in newer bodies, but the point is that there is value added from using a native higher ISO as a "starting point", and it's better if you can ETTR more by bumping the aperture/shutter speed/Exposure Compensation a bit, as long as you don't blow highlights!

Well, I'm tossing it out -- play in the sandbox and have fun!

tonylong wrote in post #12976346
If you guys want to see something really scary, try a test that will really push things.

BTW, John, good to see you here and I know I'm not posting anything new to you.

Anyway, set up a scene where you can get a "good" exposure in ISO 1600, pushed to the right a bit if you wish but with no blown highlights and no blocked shadows. Shoot the scene in Raw and in manual mode and with any in-camera settings for noise, exposure corrections and such turned off. So, what you get is the aperture and shutter speed letting in light, and the light is modified by the ISO amplification, period. The only thing of difference other than the ISO that we are concerned about is the noise collected by the sensor and then that collected "upstream".

So, we will change the one variable we can. We will keep the aperture and shutter at the same physical settings, so the same light is being collected, but we will lower the ISO (the internal amplification of the signal) to the "cleanest" ISO that Canon cameras "boast" of, ISO 100.

Now, this is 4 stops lower in Ev, "horribly unexposed", but the only difference in our settings is the ISO internal amplification! As to light, the exposure is the same!

So go ahead, you are in Manual, just lower the ISO to 100 and shoot away! After all, we all learned back in the film days that a low ISO is "less grainy" and using a high ISO "causes grain/noise", right?

And then bring your results into your Raw processor of choice. Now, it has been said that some Raw processors don't handle such things as well as others, I don't know where the lines are drawn. Raw processors use their own internal processes to handle things.

But, the idea is you now have to boost your ISO 100 shot by 4 stops to be the "equivalent" of the ISO 1600 shot. You can do this in a straightforward way with the ACR/Lightroom Raw processor. In DPP you will need to set the Raw tab to Linear processing and then boost the Brightness to the max of +2 Ev and then go to the RGB tab and boost the brightness until the levels match the ISO 1600 shot.

And now you can compare "real" ISO with the "real" native 1600 ISO.

You may be surprised! There has been a lot of analysis/evaluation that points to the fact that the lower Canon ISOs are more noisy than the higher "native" ISOs! This does tend to level out at ISO 1600, maybe more in newer bodies, but the point is that there is value added from using a native higher ISO as a "starting point", and it's better if you can ETTR more by bumping the aperture/shutter speed/Exposure Compensation a bit, as long as you don't blow highlights!

Well, I'm tossing it out -- play in the sandbox and have fun!

As far as the image sensor is concerned, there's no difference between shooting at ISO 100 and pushing it to ISO 1600 and settnig the camera to ISO 1600 (i.e. with both the shutter and aperture being the same in both cases). The difference is in the internal processing of that image from after the image data is retrieved from the sensor. Hardware/firmware in-camera processing that's done with the camera at ISO 1600 will probably be much better than the software processing that's done to the ISO 100 image after the image has been uploaded to the computer. That is the reason the key to cleaner images is to get the exposure right in the first place.

PacAce wrote in post #12976981
As far as the image sensor is concerned, there's no difference between shooting at ISO 100 and pushing it to ISO 1600 and settnig the camera to ISO 1600 (i.e. with both the shutter and aperture being the same in both cases). The difference is in the internal processing of that image from after the image data is retrieved from the sensor. Hardware/firmware in-camera processing that's done with the camera at ISO 1600 will probably be much better than the software processing that's done to the ISO 100 image after the image has been uploaded to the computer. That is the reason the key to cleaner images is to get the exposure right in the first place.

It seems to be more a case of the sensor and camera introducing lots of extra read noise after photosite amplification, such that at low ISOs the extra noise is much stronger than the amplified photosite noise. Think of a PA system with a relatively clean microphone amp, but a hiss in the final power amp, after all gain stages. If you try to record from the output speaker with another microphone, you will have a much better ratio of signal to noise if the signal is amplified more at the mic amp, so it is stronger, relative to the hiss. This is the equivalent of higher ISOs.

Of course, many other camera systems have no significant "hiss in the power amp" at all.

PacAce wrote in post #12976981
As far as the image sensor is concerned, there's no difference between shooting at ISO 100 and pushing it to ISO 1600 and settnig the camera to ISO 1600 (i.e. with both the shutter and aperture being the same in both cases). The difference is in the internal processing of that image from after the image data is retrieved from the sensor. Hardware/firmware in-camera processing that's done with the camera at ISO 1600 will probably be much better than the software processing that's done to the ISO 100 image after the image has been uploaded to the computer. That is the reason the key to cleaner images is to get the exposure right in the first place.

Yeah, the signal collected by the sensor is "exactly" the same between the ISO 100 and ISO 1600 versions, since the aperture and shutter speed are delivering the same light. In fact, a camera could be designed to deliver both an ISO 100 capture and an ISO 1600 capture -- again, the only difference being the ISO amplification.

John Sheehy wrote in post #12977220
It seems to be more a case of the sensor and camera introducing lots of extra read noise after photosite amplification, such that at low ISOs the extra noise is much stronger than the amplified photosite noise. Think of a PA system with a relatively clean microphone amp, but a hiss in the final power amp, after all gain stages. If you try to record from the output speaker with another microphone, you will have a much better ratio of signal to noise if the signal is amplified more at the mic amp, so it is stronger, relative to the hiss. This is the equivalent of higher ISOs.

Of course, many other camera systems have no significant "hiss in the power amp" at all.

I've never messed with a DSLR other than Canon, but I've heard that some like Sony are putting out gear with low noise at lower ISOs. So, "technically" you could do my little experiment and come out with equivalent results, meaning that high ISOs would have no substantial benefits, just the convenience of having a brighter jpeg/preview. Hmm, it does sound enticing!

It would seem prudent to use bracketing.

.
I am so glad that someone revived this thread. . I wish I had been aware of the thread when it was first started back in 2011, because then I was shooting with a Canon 50D as my main camera, and did not know about overexposing to reduce noise.

Overexposing and then bringing the exposure down on the computer definitely resulted in cleaner files than shooting the "correct" exposure in the first place, with my 50D. . I noticed this was also the case with my Canon 1D4, but not to the same extent. . Now that I've been shooting with a Canon 6D, I have found that overexposing and then bringing the exposure down in editing does not result in a cleaner file than just shooting the "correct" exposure to begin with.

It almost seems as if sensor tech gradually changed over the years from 2009 until 2012, and that the newer sensors, for whatever reason, do not benefit from overexposure.

I would like to know exactly HOW this change occurred. . What is it, precisely, about the newer sensors that resulted in this change? . Does it have to do with the way the data is recorded onto the sensor at the instant of capture, or does it have to do with the way the data is processed (in camera).

So I guess my question is, is this a physical change in the sensor itself, or is it a change in the software that goes into image generation?

If it is the sensor itself, what exactly did they change? . What parts of the sensor are different, and what are those differences?

If it is the software changing the way the image is generated, then what is different about the software? . What lines of code are different, and what exactly are those differences in the code?


.

Tom Reichner wrote in post #18975410
.
I am so glad that someone revived this thread. . I wish I had been aware of the thread when it was first started back in 2011, because then I was shooting with a Canon 50D as my main camera, and did not know about overexposing to reduce noise.

Overexposing and then bringing the exposure down on the computer definitely resulted in cleaner files than shooting the "correct" exposure in the first place, with my 50D. . I noticed this was also the case with my Canon 1D4, but not to the same extent. . Now that I've been shooting with a Canon 6D, I have found that overexposing and then bringing the exposure down in editing does not result in a cleaner file than just shooting the "correct" exposure to begin with.

It almost seems as if sensor tech gradually changed over the years from 2009 until 2012, and that the newer sensors, for whatever reason, do not benefit from overexposure.

I would like to know exactly HOW this change occurred. . What is it, precisely, about the newer sensors that resulted in this change? . Does it have to do with the way the data is recorded onto the sensor at the instant of capture, or does it have to do with the way the data is processed (in camera).

So I guess my question is, is this a physical change in the sensor itself, or is it a change in the software that goes into image generation?

If it is the sensor itself, what exactly did they change? . What parts of the sensor are different, and what are those differences?

If it is the software changing the way the image is generated, then what is different about the software? . What lines of code are different, and what exactly are those differences in the code? Canon has claimed changes in their code, but I won't pretend I know anything beyond that.


.

You aren't going to get anything other than guesses (educated or otherwise ) about the software unless someone works for Canon, or has bothered to reverse-engineer the code and chooses to post here.

My guesses would include improved sensors (more efficient at converting light to electrons), so one doesn't need to expose more to get those electrons. Amplifiers and A/D converters are better, less noisy, so that noise doesn't swamp low signals.

You may find this an interesting read, although it could stand some updating:
https://clarkvision.co​m …rmance.summary/​#full_well

EDIT: You may find this Canon link interesting, especially the part about noise removal at the pixel level (you may need to scroll down to see it): https://global.canon …/s_labo/light/0​03/05.html

I shoot a 6D also. I underexpose at night and bring the shadows up...within reason. I try not to let the ISO go above 6400 because that's where things get kind of gross looking, lol.