How-to: Making an underexposed low-ISO Canon image look like an underexposed low-ISO Sony image

There has been a lot of talk about the dynamic range and underexposed low-ISO performance of Sony cameras due to a sensor design that Sony has. This has been disputed as an advantage that Sony circuitry has in the way the bits from the photodiodes are processed and manipulated before the data is written to the ARW raw file, not a better sensor that Sony makes. Since we know that there is nothing intrinsically better about the Sony sensors besides the circuitry that processes noise handling, we thought we sould show everyone how to generate the same underexposed low-ISO, high dynamic range quality in a Canon image. Below is the step by step process using free software. Soon there will be a pattern noise program to automate this process. But for now, here is the tutorial for any critical images that you need to recover.

You will need to download a program called FIJI to create the filter and process the image. It can be found at

http://fiji.sc/Fiji

Here is our underexposed image:

Although using the Fast Fourier Transform as a filter to remove specific low spatial frequency noise problems is very useful, it will also remove any wanted signal at the same frequency. So will like all noise filters have an unwanted effect on certain elements of the image. Also the truth is that using FFT in this situation is never going to be a simple one step solution. It is theoretically possible to produce a FFT filter for each individual camera body using a dark frame exposure, that fails to take into consideration that the actual noise present is proportional to the relative level of exposure. So we are back to having to build the FFT filter from scratch for every exposure

I would also strongly disagree with your statements on the advantages provided by the Sony sensor design over the more traditional sensor designs that Canon are still using. Yes at the level of the sensor sensels the noise introduced, remembering that of the effectively three dimensions that the sensor is recording, it is directly digitising the signals in two of those dimensions. So it is only the intensity that is being recorded in an analogue manner from the sensor. As these signal voltages come out of the actual sensel on the sensor unit the SNR is going to be about the same for any sensor of similar dimensions. Sensel area will define how much of an effect the random arrival of the photons adds to the noise in areas of constant tone. The same will also apply, but in reverse to the amount of quantisation noise might be introduced in areas of high image detail. There are other sources of noise as well which are also going to be fairly constant regardless of manufacturer, for any particular basic sensor technology.

It is the next stage that Sony has the advantage over other sensor manufacturers. The signal is passed to an analogue amplifier stage which is used to produce the different ISO equivalent values. Generally for base ISO the gain is one, meaning there is no amplification. For each stop of ISO gain the analogue gain is doubled, so assuming a base ISO of 100, that would have a gain of 1, ISO 200 would have a gain of 2, ISO 400 would be 4 and so on. Remember though that you are doubling both the signal, and the noise, so eventually you get to a point where the signal is so small that the inherent noise (the noise floor) is bigger than the signal. This is the point where you stop increasing the analogue gain, as it no longer gives you an advantage. From the amplifier the signal is passed to the Analogue to Digital converter, where it is converted into a number between 0 and 16383 (for current 14 bit ADCs) proportional to the number of photons arriving at that sensel during the exposure time. For Base ISO that proportionality is about 1. The issue is that in passing the signal from the sensel to the amplifier and from the amplifier to the ADC the signal passes along conductors. Unfortunately these conductors act as little antenna, and so pick up radio waves, which add small amounts of voltage to the signal. The longer the conductors, the more noise will be collected on route. What Sony have done is to move the analogue amplifiers and the ADCs on to the same bit of silicon as the sensels. This significantly reduces the length of the analogue interconnections, and so significantly reduces the amount of noise introduced in to the signal.

It seems that Sony have a Patent for this idea, although integrating circuitry has long been a common aspect of electronic design. There are also drawbacks to adding additional complexity to any particular chip design, the more components that you have, and the larger the area of the chip, which are usually directly linked, the higher the failure rate during production. One reason that FF sensors will always be significantly more expensive than crop sensors is the size. Double the area of a chip and you are four times more likely for it to have a fault. So for Crop to FF which are 2.56 times the area of a crop sensor the failure rate will be 6.5× higher. So adding extra components to the sensor chip is always a risky proposition.

As far as an optimal design for a digital sensor chip you would want to be able to build the chip with the sensels on the front surface, and then use the rear surface to build both an analogue amplifier, and ADC unit directly behind each sensel. This would produce a sensor with the best possible SNR using current silicon based technology. The down side would be massively more complex silicon designs, which the attendant increase in production failure rates pushing the costs.

So the advantage for the Sony sensor is not that it has a clever system to reduce the noise, it simply adds less noise to the signal. No system of noise reduction will ever be as good as NOT adding the noise in the first place.

Alan

Is it just me, or would this thread make more sense with a shot of the 'improved' final image in it?

BigAl007 wrote in post #17326053
So the advantage for the Sony sensor is not that it has a clever system to reduce the noise, it simply adds less noise to the signal. No system of noise reduction will ever be as good as NOT adding the noise in the first place.

In view of the fact that Nikon uses the Sony sensor, what explains the difference in noise levels of the Sony vs. Nikon, as mentioned in DPReview.com ?

GeoKras1989 wrote in post #17326066
Is it just me, or would this thread make more sense with a shot of the 'improved' final image in it?

I think OP intended to post a string of posts in sequence, as he had posted four total posts, the last three of which were place holders. I saw what he was apparently doing and looked forward to coming back and reading the rest of the series, when updated. Methinks a moderator decided the place holders were verboten and cleared them (or maybe the OP changed his mind).

I think that there are probably many things that will cause variation in output between different implementations of the same basic sensor technology, such as the range of Exmor sensors. As I understand it they make Exmor based sensors in a variety of pixel densities. If we were to normalise to Full Frame that gives a range of pixel densities from 12 Mpix to 54 Mpix. That is a variation of more than two times (2.16×) in linear resolution. The high density sensor will have lower spatial quantisation errors, but will have much higher levels of error due to the random arrival of the photons, which is far more important when dealing with the darker areas of the image. Also Nikon use different processing systems once the digital signal is off the sensor. RAW images should be reasonably close, but I would expect to see significant differences in OOC JPEG images for the different brands. Even where cameras from Nikon and Sony have the same size sensor, and the same pixel resolution, that is still not a guarantee that the sensors in them are identical. Canon have had a range of 18 Mpix crop format sensors, and although they are all quite close in performance there are enough differences to say that they are not all identical. Even the 70D and 7DII would seem to have slightly differently engineered sensors. otherwise it would be difficult to explain the apparent improvement to the banding in 7DII image files.

Also consider that the Exmor sensor design brings the improvements to the sensor at base ISO only. From the accounts that I have read once you up the analogue amplification the advantage swings to the Canon sensor, which is superior from about ISO 1600, due to differences in the way that noise is generated in sensors.

Alan

I was going to put a tutorial in here on how to build a fast fourier tranform filter, apply the filter, and then reconstruct the image with an inverse fast fourier transform. But the problem is that I lost my placeholders and I can only add 2 images per post. I'll have to redo this with pictures of my computer screen and the progress of the image by hosting the shots on flicker.

In the meantime, here are all the steps: I was going to have a picture to go along with each step:

0. (Optional) rotate the image clockwise or counterclockwise by 5 degrees. This will offset the pattern and make it easier to eliminate completely.
1. Open the noisy image from the Canon camera in FIJI (if there is no color to the noise (there usually isn't), then optionally convert the image to the Lab color space, and apply the FFT to the "L" channel only, don't apply the process to the "a" or "b" channel - however you don't have to do this in the Lab color space and this procedure will work well in RGB or CMYK either way).
2. Open the Process >FFT>FFT menu item in FIJI. The image will be processed with a fast fourier transform.
3. Save the resulting image as a TIFF or JPEG. It will be a grayscale image with stars, crosses, lines, or dots, depending on the noise pattern you're trying to eliminate.
4. Open the FFT image you saved from Fiji in Photoshop. If the contrast is low, then apply a "Levels" adjustment to increase contrast of the FFT image so you can see the white dots, stars, crosses or lines easier.
5. Once you have the image open, make a new transparent layer above it and paint on the transparent layer in black on the areas of the FFT image that you ultimately want to suppress. The areas you want to suppress will be white spots, crosses, stars, or lines. Use a very small and a very soft brush to paint the dots, stars, crosses, or lines black. You can also use the rectantular tool for lines, perform a gaussian blur to the selection, and then fill the selection with black.
6. Once you have all of the lighter star/cross/dot areas painted out in black, put a blank layer between the FFT image and your painted transparent layer.
7. Fill this new layer with white.
8. Now your paint brush marks are composited onto a white background (white reveals, black conceals). It should look like a white image with all of the black marks that you created.
9. Save this black-and-white composite image as a new TIFF or JPEG image. We're going to designate this as the FILTER image.
10. Open the composite filter image we just saved in Photoshop in Fiji.
11. Open the original noisy image in Fiji (the one you used to make the FFT in the first place in step 1).
12. Click on the original noisy image.
13. Create the custom filter by choosing Process>FFT>Custom Filter
14. Select the black-and-white composite filter image that we saved in Photoshop (we are using the Process>FFT>Custom Filter menu item on the original noisy image that you're trying to fix and designating the filter image you just made in Photoshop as the filter).
15. The noisy image will now have the Fast fourier transform filter applied to it, then Fiji will apply the filter to the resulting fast fourier transform, and then an Inverse fast fourier transform will be applied. The filter you created should eliminate the pattern noise you saw.
16. If there is still lingering noise, then just repeat step 2 on the cleaned-up image to see if you missed a few spots. Open up your FILTER image in Photoshop and add some additional black marks where they need to be placed. Then continue with step 10.


This takes about 5 minutes to do.

Question:
After step #4, after opening the FFT in PS, wouldn't it be easier to then just invert it and apply a blur?
Or duplicate the FFT layer first and invert the duplicate to be safe.

CRCchemist wrote in post #17325814
Since we know that there is nothing intrinsically better about the Sony sensors besides the circuitry that processes noise handling

I wasn't aware that we knew this. How it is that we know this?


Looking forward to seeing the results.

I can't say that I'm impressed.

CRCchemist wrote in post #17328523
"Question: After step #4, after opening the FFT in PS, wouldn't it be easier to then just invert it and apply a blur? Or duplicate the FFT layer first and invert the duplicate to be safe."


No, you don't want to do that, because then you'll be filtering out good image data in the middle of the filter, so you have to knock the stuff out that isn't in the center, such as the stars, dots, lines, and crosses. But I think you're on to something. I might try your idea, but create a mask in the middle and along the axies. I think your idea might make this go a lot faster.

I was thinking it would probably save a whole lot of brush strokes letting PS do most of the work!

WhyFi wrote in post #17328565
I can't say that I'm impressed.

You and me. Not worth all that effort.

Did I miss the final picture? I just want to see start -> finish. I will let you experts handle the -> part

Is the 2nd pic in the OP the final?

thefranklin wrote in post #17339928
Did I miss the final picture? I just want to see start -> finish. I will let you experts handle the -> part

Is the 2nd pic in the OP the final?

Looks like the post with the 'final' was deleted. To me, there was still a discernible pattern to the noise and the detail was really lacking. I don't think that it was in any way equal to the results that I've seen from RAW files from Sony sensors.

I'm not sure why you'd say that Why Fi... the pattern noise was eliminated 100%. That post-production procedure I wrote out completely eliminated the pattern noise. I didn't give you a procedure to eliminate all of the other noise that is inherent in an image (that a Nikon and Sony also has.) I only told you how to eliminate the horizontal and vertical pattern noise that shows up when you increase the brightness of the image by several stops at low ISO.