Zero Noise - interior shot with high DR

[kirkt]

So, I have been experimenting with Zero Noise (http://photography-on-the.net/forum/showthread.php?t=763460) and I have found this app to be a gem. TO be able to pack a ton of noise free data into a single 16 bit TIFF, with no noise in the shadows is really a revelation. While my experiments have not been aesthetically impressive, I have tried to create some really difficult imaging situations to see what kind of latitude I can muster from the image data in post. It really is a great thing to be able to apply a horrific curve, and then another horrific curve and not destroy the histogram.

Here is one of two "case studies" where the dynamic range is mid to high. I tried to include a bunch of different surface types, textures, colors, etc. all lit by the natural window light of afternoon, facing north - with the bonus of the satellite radio LCD - just because. :)

I combined two RAW files with Zero Noise to get the base 16 bit TIFF. Here is a composite of the two RAWs, converted to TIFFs mapped to gamma 2.2, that were combined to make the base 16 bit TIFF:

http://kirkt.smugmug.com/photos/677647962_UYUSb-XL.jpg


Here is the base TIFF that is generated by Zero Noise, no processing:

http://kirkt.smugmug.com/photos/677671778_7teuU-XL.jpg

You can see that the base TIFF is mapped to the shortest exposure luminance values, so it appears dark, with the brightest areas exposed properly (ie, outside the window) - notice how it looks essentially similar to the darker of the two combined images in the first figure. Ahhh, but here is where the coolness begins....

TO get a tonemapped version of the data, I usually had to apply two major league curves to redistribute the histogram appropriately. In the above base TIFF image, the area boxed in red is the area in darkest shadow. Here is a 100% crop, with a crazy curve applied to bring up the level of the shadow area - note the lack of noise:

http://kirkt.smugmug.com/photos/677671766_CKKYU-L.jpg

Incredible! That detail is not even present in the darker RAW exposure used in the blend, and that RAW is ravaged by noise throughout most of the shadow tones. The ZN base TIFF is virtually noise free and full of data in the shadows. Fan-freaking-tastic.

Here is the massaged final version of the image, processed in PSCS4 from the base 16 bit TIFF - there are some blending artifacts, due to operator error, I am still learning:

http://kirkt.smugmug.com/photos/677647946_FnXE8-XL.jpg

Very, very cool. The images are combined through a narrow progressive blend, so the details are preserved (ie, the pixels from each image are not averaged together, so small shifts in the camera and microregistration errors are minimized). The output gives you plenty of room for tonemapping and you don;t need to use details enhancer type tone mappers that produce halos and color shifts. THe blending is similar in concept to Enfuse, but produces a 16 bit set of data suitable for tonemapping, instead of an 8 bit tonemapped (blended) exposed output file like Enfuse. More flexibility, all with minimal noise.

I encourage you to try this exercise yourself and experiment with this remarkable approach to increasing the dynamic range of your data, without the side effects of the typical HDR workflow. I am still trying to learn how ZN works, but it is clear that this tool is a powerful way to extend your camera's sensor.

ENjoy!

Kirk

[kirkt]

Here is a second case study, with really weird natural light reflecting from the yellow walls, combining with the higher temperature bluish daylight. I chose black lenses as the subject to see what detail I could pull out of the shadows. Wow.

Comp of the two RAWs that were combined:

http://kirkt.smugmug.com/photos/677647873_EQvAg-XL.jpg

Final tonemapped version, based on the base 16 bit TIFF generated from Zero Noise:

http://kirkt.smugmug.com/photos/677647903_PD3TW-XL.jpg

Natural light, northern exposure, late afternoon. There are some noticable halos around the chair legs, especially, due to my experimenting with Shadow/Highlight and HiRaLoAm sharpening. More experiments necessary. ;)

Here is a 100 percent crop of the deepest shadow tones, mapped to the above final level. The noise is tolerable, all things considered.

http://kirkt.smugmug.com/photos/677739596_RhjxU-L.jpg

Kirk

[Gary McDuffie]

Your zero noise work is amazing, Kirk. Wow!

[Picture North Carolina]

First, a very excellent and noteworthy white paper. Thanks for taking the time to experiment and post.

The lack of noise in that first 100% crop (dishrack / utensils) is rather shocking. And if that were tonemapped by photomatix, it would probably have noise as big as grains of sand.

One thing I'm curious about, tho. Take a look at the first final blend (kitchen) scene. Look at the window, and the window frame on the left. There is a vertical line there which is lighter and whiter on the right, and darker with more yellow on the left. The same vertical line appears on the right side of the cabinet which is left of the window.

If you look at the original, unblended shot (f11 / 4.0) you can see that those vertical lines (in the blended final version) correspond to the shadows in that original unblended image.

Is this one of the artifacts you mentioned? It appears as if in the blending process, those areas have been reversed - i.e., the dark shadow is now lighter than the surrounding area that was originally brighter (but is now darker).

[kirkt]

Hi CannedHeat - yep those are the artifacts - this has to do with my not having the nuanced understanding of the threshold setting associated with the blending between images. You are right on the money, and this glaring example that you pointed out is the pilot error to which I was referring. There is a slider that permits one to adjust the saturation level at which the blend occurs and I have to start experimenting with that. Because I am only blending two images, if the threshold level occurs in the middle of a conspicuous area, like the one above, the blending parameters need to be massaged, I suppose. You have hit the nail on the proverbial head of my next round of experiments. I am also experimenting with the choice of images and the number of images, which may also help the blend.

The replacement of the noisy shadow areas in the first RAW with the rather noiseless second set of pixels is a clever approach, isn't it?

I would post the grayscale blending map to give you an idea of how the blend was computed, but my hard drive is currently in the process of taking a major dump, requiring reformatting, so, I'll see if I can recreate the mapping later. I suppose writing and rewriting and editing and saving 73MB TIFFs can run a poor little hard drive into the ground.

The upshot is, for a two image blend, the areas in the transition region will either be from one image or the other and there may be a discontinuity at that boundary, especially noticeable if that boundary occurs in the middle of a smooth gradient, like the shadows on the cabinet side panels above. I will work on this and report back.

Thanks CH and Gary for your comments and encouragement.

Kirk

[Picture North Carolina]

Hi CannedHeat - yep those are the artifacts - this has to do with my not having the nuanced understanding of the threshold setting associated with the blending between images. You are right on the money, and this glaring example that you pointed out is the pilot error to which I was referring. There is a slider that permits one to adjust the saturation level at which the blend occurs and I have to start experimenting with that. Because I am only blending two images, if the threshold level occurs in the middle of a conspicuous area, like the one above, the blending parameters need to be massaged, I suppose. You have hit the nail on the proverbial head of my next round of experiments. I am also experimenting with the choice of images and the number of images, which may also help the blend.

The replacement of the noisy shadow areas in the first RAW with the rather noiseless second set of pixels is a clever approach, isn't it?

I would post the grayscale blending map to give you an idea of how the blend was computed, but my hard drive is currently in the process of taking a major dump, requiring reformatting, so, I'll see if I can recreate the mapping later. I suppose writing and rewriting and editing and saving 73MB TIFFs can run a poor little hard drive into the ground.

The upshot is, for a two image blend, the areas in the transition region will either be from one image or the other and there may be a discontinuity at that boundary, especially noticeable if that boundary occurs in the middle of a smooth gradient, like the shadows on the cabinet side panels above. I will work on this and report back.

Thanks CH and Gary for your comments and encouragement.

Kirk

The point is that you have created in me a great interest in this software. Hopefully, when time allows, I'll start farting around with it.

As to the blend delineation lines, (and if the app allows it) as you say different, or more, input images may resolve the problem. Even if it does not, I could see where producing two different blended images, layered in photoshop, using a mask to paint in smoother transitions would easily solve the problem. Good luck.

[kirkt]

The workflow within ZN is such that, prior to the actual blending, a grayscale blend map is generated - you then specify this map as the blend map for the actual blend. I am going to see what happens if you hand edit the blend map prior to specifying it for use in the actual blend, and see if that level of hand massaging will be respected during the blending process. This is similar in some respects to the way Enfuse works, in that you can embed an alpha channel in the TIFFs you combine in Enfuse, where, for each image you are trying to blend in Enfuse, the alpha channel within the image overrides any of the Enfuse blending - thus you can custom specify which pixels to accept or reject on a per-image basis prior to the blending operation.

In ZN, there is a single blend map, where each gray value corresponds to a particular image in the set you desire to blend - in the blend, the pixels from that donor image will be used where the map shows the gray value associated with that donor image. This, I assume, can be edited by hand to tweak the blend. I will give it a shot and report back.

Blah blah blah. :)

Kirk

[_GUI_]

TO be able to pack a ton of noise free data into a single 16 bit TIFF, with no noise in the shadows is really a revelation.

Hi Kirk, don't be surprised that a 16-bit non-linear TIFF can encode a HUGE dynamic range if the gamma curve is cleverly used. HDR formats are usually floating point based, but just because it is simpler to handle the values in a linear fashion for post processing (e.g. for tone mapping algorithms).

With the needed number and spacing of source shots, you can encode a crazy DR in a single TIFF. Look at this linear vs 2.2 gamma table showing the number of levels the TIFF can encode on each stop:

http://www.guillermoluijk.com/article/superhdr/tabla.gif

In the 16th stop we still can have up to 157 native differentiated levels, vs just 1 in a linear encoding.

To test this, I shot a scene with a huge DR of >16 stops making 5 shots 3EV apart:

http://www.guillermoluijk.com/article/superhdr/resultado_lite.jpg

And ZN put all that information into a single TIFF that can be pushed +15EV without any noise or posterization appearing (yes, I said 15):

http://www.guillermoluijk.com/article/superhdr/progev.jpg

Please download this resulting superhdr.tif (http://www.guillermoluijk.com/download/superhdr.tif), and do insane things to it (increasingly +3EV curves are already included). It blows before any noise or posterization arises. The original article (SP) here: TIFF WITH MORE THAN 16 STOPS OF DR (http://www.guillermoluijk.com/article/superhdr/index.htm)

The difficult part is how the hell to tone map that huge captured DR, but the information is there. In general, trying to display information spreaded along more than 12 stops in the real situation, ends in unnatural looking images. It's better in these cases to allow some highlights blown and clip some deep shadows to black, so that you get a fairly good contrast in the middle range.


In ZN, there is a single blend map, where each gray value corresponds to a particular image in the set you desire to blend - in the blend, the pixels from that donor image will be used where the map shows the gray value associated with that donor image. This, I assume, can be edited by hand to tweak the blend. I will give it a shot and report back.

Correct. In fact a quick edition of the blending map only in the affected area, is what allows to use ZN even on non 100% static scenes for anti-ghosting. For example, the original blending map cut the legs of this guy:

http://www.guillermoluijk.com/tutorial/zeronoise/ag2_antes.jpg

After a quick manual edition of the blending map (just painting white):

http://www.guillermoluijk.com/tutorial/zeronoise/ag2_despues.jpg

http://www.guillermoluijk.com/tutorial/zeronoise/zn.jpg

http://www.guillermoluijk.com/tutorial/zeronoise/comp.jpg

The cost of the operation was obviously some more noise in the area painted in the blending map.


If you want to have some fun, look what Photomatix managed to produce with the same pair of RAW files used in ZN (mov1.cr2 (http://www.guillermoluijk.com/download/mov1.cr2) and mov2.cr2 (http://www.guillermoluijk.com/download/mov2.cr2)):

http://www.guillermoluijk.com/tutorial/zeronoise/pm.jpg

The 4+1/3EV exposure gap (http://www.guillermoluijk.com/tutorial/zeronoise/histmov.gif) was too much for Photomatix. Funily you can take the TIFF produced by ZN (which already contains all the DR, is sharp and avoided the ghosting), create several replicas of it at different exposures, and feed them into Photomatix. Then the result of Photomatix will be correct. Disgusting as usual, but correct.

Regards

[kirkt]

Right on Guillermo - I read that article about the gamma 2.2 encoding and, like I said, it is a revelation. THe tonemapping part is sort of crazy, in that one has to more or less discard the traditional notions of the kinds of curves adjustments that are suitable in PS - I find that I apply some crazy curves at the beginning and the histogram is still happy and healthy. It is going to take me a while to start thinking outside of the traditional post-processing, but I am liking what I have gotten into so far - each time I think I am about to destroy the histogram, it stays intact. So, with traditional tools used in new ways, the post processing becomes one of thinking beyond the traditional limits but within the comfort zone of tried and tested tools.

Thanks for the commentary, this is really helpful and really cool.

Kirk


BTW, here is the same shot in Case Study 1, but blended from 4 images (2 additional images inserted into the blend between the original 2 blended above):

http://kirkt.smugmug.com/photos/678445422_win3Y-XL.jpg

And here is the blend map, generated from the defaults in ZN:

http://kirkt.smugmug.com/photos/678445455_dUbPn-XL.jpg

I tried not to neutralize the daylight-filled shadow regions too much, so that one could better visualize the transition in the shadow (ie, I left the daylight filled regions bluish). Much better in terms of the automatically generated blend map in the previously cited problem area - also note that, in the final PP'ed version back in the first post, the mini pumpkins (especially the left-most one) also had some weirdness going on at the boundary between the shadow and midtone/highlight regions - this is also remedied in the newer blend posted in this entry. Note that the blend map is the image prior to straightening and slight content aware scaling necessitated by the straightening rotation. It does not register one-to-one with the final image for this reason, but is close enough to make a visual comparison.

[kirkt]

Oh for crying out loud - I just realized somewhere along the way in the last couple of weeks that I must have f'ed my monitor calibration - forgive me if these images are totally off. I have to recal when I get home and compare to my Cinema Display (I'm on a MBP at work).

Crapola. Kirk

[Picture North Carolina]

Perhaps a little tinted on the blue side, but the miracle here is not calibrated monitors but noise control. If you can show me a way to calibrate my monitor in such a way it reduces noise on the output image, I'll buy you a beer! :)

[kirkt]

Apparently, I have been struggling with Snow Leopard and ColorSync like a others. I do most color critical things in Leopard on a Cinema Display/Mac Pro, where ColorSync and all that jazz still plays nice. The bluish tint is just the daylight, but, for example, if I capture a screen grab of the color histogram channels and then convert that screen grab to sRGB profile for posting here, the blues go violet. Egad. I will post an example here and see if you can appreciate my problem - as always, I value the feedback. My calibration issue is unrelated to ZN, but more related to my enthusiasm for trying to break new things like computers and software. :)

kirk

[_GUI_]

The bluish tint is just the daylight, but, for example, if I capture a screen grab of the color histogram channels and then convert that screen grab to sRGB profile for posting here, the blues go violet.

One of the collateral advantages of working with the unprocessed underexposed images output from ZN, is that colour casts become very apparent before any curve is applied. They are not wrong colours, they are real colours present in the scene, relative to the WB'ed area. We didn't notice them when looking at the scene because our eyes get blind in such luminous spots, so colour perception is reduced.

When lifting curves are applied, the colour tints start to desaturate (then tend to (255,255,255)) so it's more difficult to detect and accurately cancel them.

Look at this scene, WB was done over the red square on the wall (i.e. ZN forced that area to be middle gray). A lot of colour casts became visible:
- Daylight magenta (curtains)
- Blue tubes (ceiling)
- Purple tubes (over the staircase)
- Warm tungsten (the lamp)

http://img230.imageshack.us/img230/4492/10391398.jpg

After some pp on them the colour casts went away. Perhaps it would have been a good idea not to completely eliminate them.

http://img230.imageshack.us/img230/9341/25162169.jpg


To do more than one blending using different white balances and then manually mix them in PS is totally possible. I did some tests to define more than one white balance in the same scene in ZN, but it was more difficult to code such algorithm than the advantage obtained.

BTW the picture is crap, just a test.

Regards.