This is a little tutorial on building a mask to protect edges from noise reduction software.
NOTE: This tutorial seems to require noise reduction software that can be run as a Photoshop Plugin. But at the end I list steps to put this tutorial to use when you use a stand-alone noise reducer. You can still get back your detail!
And this whole masking process has a lot of uses, so you might wish to read on anyway.
Noise is most apparent in open areas that don't show much detail. Details in an image will generally hide noise, or at least noise isn't so apparent. So it's a good idea to protect the detail because NR software will remove detail along with the the little noise speckles. This loss of detail can soften the edges, so we're going to protect them with a mask. The NR software will run on the rest of the image, leaving the detail - the edges - untouched or barely touched.
First we'll start with a noisy image. This is a 100% crop of a finch who stumbled near my bird feeder that I have conveniently placed within range of a window. Alas, this day was overcast, so I had to bump the ISO to 1600 to get a decent shot.
Kinda ugly. But you'll notice that the noise is most apparent in the open section, the background, as well as the plain colored beak. The edges of the feathers don't show much noise because the details of the feathers help hide the noise. But yet the open areas of individual feathers show noise.
To build our mask we first want to flip to the Channels tab and click on each of the channels, Red, Green and Blue. We're looking for the most contrasty channel - the one with more apparent details. In my case it was the green channel. So I'll right click on the Green channel and duplicate it.
A pop-up box will, amazingly enough, pop up and ask you for a name. I stretched a bit and chose "Mask" which I thought was a good choice. The Mask channel will be selected now, and you'll see a grayscale version of the bird.
In order to make the mask to protect the edges we first will find the edges. Choose Filter... Stylize... Find Edges.
I now have this:
Yep, it found all the edges, and also found all the edges of the noise, too. Not to worry, as we'll fix that later.
Also, all the edges are kinda dark grey, which isn't ideal. In Photoshop a mask is a grayscale image, with Black designating the areas to be completely masked (protected) and White designating areas not to be masked at all. Shades of gray in between designate the amount of masking to be done.
This is a very important concept here. Black areas will be completely masked and any noise reduction you do to the image will not be done AT ALL to these areas. White areas will get noise reduction at full power.
So of course we want the edges to be very black while everything else is very white. So we want to increase contrast in this mask to drive everything to black and white. We'll do that with Curves.
Bring up Curves (Image... Adjustments... Curves, or Ctrl-M). Click on the little dot in the lower left corner, the Black Point, and drag it towards the middle but keep it at the bottom. Now click on the upper right corner and drag the White Point to the middle at the top. Your "curve" should now be a straight light going up and down.
If you check out the image you'll see that it's now pure black and pure white, but the lines are very thin. They're not wide enough to cover the detail completely - they'll only cover the center of the detail. We want the lines to be wide enough to completely protect the edges. So grab that white point and drag it back towards the right. The lines will get a little wider...
I chose this:
Which got me this:
Note that most of the speckles in the background are gone. We're still going to fix this later.
Each image will be different though. An image of a bird has a lot of small details so I want a lot of thin lines - the mask is just as detailed as the image. If you had a picture of a car, with less but larger details, you'd want less but wider details in the mask. So play with the Curve depending on your image.
Right now my bird looks good.
Except for two things: the speckles in the background, and the fact that this mask would make for very hard edges. We don't want hard edges, - it looks better if you smooth the change from complete NR (white) to no NR (black). We're going to make the transitions smoother by blending from white to black - yep, were' going to add some shades of gray.
Then we're going to get rid of those ugly speckles. No worries.
To add the shades of gray we're going to blur this mask using Gaussian Blur. Choose Filter... Blur... Gaussian Blur. I chose a Radius of 1, which is about the limit for this image because more would start to blur the lines together. We don't want this, so 1 works for this image. (Again, with a car you might want 2 or 3 or...)
I got this:
Now to get rid of the speckles. Click on Filter... Noise... Despeckle. No options, and it got rid of some of the noise, but not much really. I could have run this early - before the blur - but I feel that it can destroy too much detail in the mask. It's up to you and your image.
But the noise is unacceptable here, so I'm just going to grab the Pencil tool and "erase" the speckles with white. I use the pencil here because it uses a hard edge, rather than the soft anti-aliased edge of the Brush. Just my preference.
Erasing the speckles this way is easy because I have such a large, clean background. Your image may be different, and erasing may be a pain. So try undoing a couple steps and try running Despeckle before the Blur. It all depends on the image.
So far, I now have a blurred despeckled mask, but it's too gray. I want Black to completely protect the edges. So back to Curves to increase contrast. But this time I'm just going to drag the black point over towards the middle and leave the white point alone. If you drag it too much you'll drive all the grays to black and detroy the smooth transitions, so don't go overboard.
I choose this Curve:
Which gave me this:
Looks good. Even better, we're almost done.
Click back on the Layer tab, and click on the Background layer, and you'll be back to the original image.
? ? ?
No worries. The channel is saved, we never deleted it. Now we just want to load the Mask channel as a selection. Simply click on Select.. Load Selection.. and chose the Mask channel. Suddenly the image is covered with those annoying marching ants.
Since they're annoying - and hide too much because they're in the way - hide the ants by pressing Ctrl-H. You're back to the original image, but the selection is hidden, so now you're ready to run the noise reduction software.
I use Neat Image, and I run it a little stronger than normal because I know that the important parts are protected by the mask. By the time the NR is done, I get a nice looking image.
Here's a before and after:
(Note that it hasn't been sharpened yet.)
As a comparison, here's the original next to a copy where I ran NR without a mask. The image on the right is what I used to get, and the NR is about 20% less than used above!
So this was an mask used to protect the edges during noise reduction. Think about sharpening for a second... You generally want to sharpen only the edges and not the open areas, right? This same process can do it, but after you run Find Edges simply Invert the image using Image... Adjust... Invert. You'll end up with a mask that will protect the open areas from sharpening, and the sharpening will just run on the edges! Cool!
Now what if you run a noise reduction program that's standalone, one that can't run as a PS plugin? Well, this tutorial can *still* work for you with a couple extra steps.
You'll need 2 copies of the image - the original, and one where you ran noise reduction outside of Photoshop.
1 ) Open them both in Photoshop.
2 ) Run this tutorial on the NR'd version to get the edge mask.
3 ) On the original, click on the Background, hold the mouse button down and drag the background to the NR version.
4 ) The NR version will now have 2 layers with the original on top.
5 ) Click on Select... Load Selection... Mask.
6 ) Ctrl-H to hide the selection.
7 ) Hit the Delete key.
8 ) On the top layer (the Original image) click on the Eye icon in the Layer palette. This will turn the layer off. Click again, and turn the layer on. You'll see the nice noise reduced image, but with the top layer on some of the detail will come back. If it's difficult to see the difference re-run the tutorial and make the mask stronger (blacker).
Now all that seems like a lot of work. It's not so bad once you run it a few times. But Since you read through this whole tutorial I'll give you a bonus - my Action for this.
Right click and save this to your Actions directory, which for me is C:\Program Files\Adobe\Photoshop CS\Presets\Photoshop Actions. Go back into PS, click on the Action tab, then the little triangle to the right of the Actions tab. Click on Load Actions, and choose NRMask.atn. That's it, you now have my action for the above tutorial. Use it freely, modify it freely, give it to your friends.
Some of the steps in the action, like despeckle, are automatic and just run. Others, like all the Curves, wait for your input which is explained above. If you don't like my choice for Gaussian Blur, expand the action and click on the empty square next to the checkmark. Now theaction will pause and wait for you to change the radius.
But please, if you cheated and just came to get the action you'll be cheating yourself. Read the tutorial to understand what is going on. Then you can run the action with an idea of what you need to change.
NOTE: As always, Photoshop gives you many ways to do things. There *are* easier ways than this, but this is what I developed as I was stumbling through it. Hopefully other, more experienced Photoshop users will add more tips to make this a better, easier process.