How Raw is RAW?

[Mike H]

I'm a bit confused about RAW images.

As I understood it, RAW images are unaltered data harvested from the sensor. They allow the user to "process" the image, i.e., set the contrast, density, and color balance of the image using the computer and specialized software. Their strong point is not throwing away any data, allowing you to work with every bit of data that the camera has captured without the camera making any changes to the data.

But consider this: if you shoot two frames exactly the same way except that you change the color balance setting from one frame to the next, you will get two clearly different image files. So hasn't the camera altered the file depending on what you told it about color balance? Is the RAW file really a raw file? It would appear not.

Why do I care? Aside from enjoying knowledge itself, I also want to know what my camera is up to for practical reasons. If the camera really does do some processing when it makes RAW files, we need to understand why, how, and when it does this to control our workflow.

Erudite comments are welcome. Thanks.

Mike H

[timmyquest]

No, it is still a RAW image.

A good way to look at it can be found in last months Photo Magazine.

When you take a picture your camera collects certain data, the image its self, colors, WB, etc. When you shoot Jpeg all of these things are applied to the image and saved as a compressed file (think of the flaten action in PS).

When shooting RAW you can still change all of these atributes as they havnt actually been applied into the image.

[Mike H]

Thanks for the reply, Timmy.

Perhaps it really isn't altered, and what I'm seeing on the back of the camera when I change the white balance are JPGs created to give review images. That is, you're not really looking at the RAW file on the review screen?

Mike H

[scottbergerphoto]

The only actual processing that is done to a Raw file is a proprietary amount of sharpening, amount known olny to Canon, that occurs even with sharpening turned off in the camera.
Source: Chuck Westphall, Director of Technical Information, Canon, USA
Regards,
Scott

[Quinn Porter]

Thanks for the reply, Timmy.

...what I'm seeing on the back of the camera when I change the white balance are JPGs created to give review images.
Mike H

That's right. The camera displays the embedded JPEG on the review screen, and that JPEG does have the white balance setting applied. The RAW file contains the white balance information from the camera, but it is not applied to the RAW file. Rather, when you view the RAW file in RAW conversion software, the white balance info is applied by the software. However, it's applied in such a way that it is not permanent until you actually convert the RAW file to a JPEG or another format.

[CyberDyneSystems]

Also,

You RAW viewer utility WILL display the adjusted white balance from one image to the next.. but the WB info is not "IN" the raw data.. it is alongside it if you will.. sort of part of a set of instructuions that tells the software what your camera settings were..

All of this info in the "instructions" will usually be displyed in the viewer by default.. but it can all be tossed out to.

The presence of this additional info has no direct effect on the RAW data.. it is in addition to it and can be used, or ignored.

[robertwgross]

That's right. The camera displays the embedded JPEG on the review screen, and that JPEG does have the white balance setting applied. The RAW file contains the white balance information from the camera, but it is not applied to the RAW file. Rather, when you view the RAW file in RAW conversion software, the white balance info is applied by the software. However, it's applied in such a way that it is not permanent until you actually convert the RAW file to a JPEG or another format.

I believe that is correct for some Canon digital cameras, but not all. On some models, there is no embedded JPEG in the RAW file.

---Bob Gross---

[Pekka]

That's right. The camera displays the embedded JPEG on the review screen, and that JPEG does have the white balance setting applied. The RAW file contains the white balance information from the camera, but it is not applied to the RAW file. Rather, when you view the RAW file in RAW conversion software, the white balance info is applied by the software. However, it's applied in such a way that it is not permanent until you actually convert the RAW file to a JPEG or another format.

I believe that is correct for some Canon digital cameras, but not all. On some models, there is no embedded JPEG in the RAW file.

---Bob Gross---

1D series are totally unsharpened in RAW, consumer models are slightly sharpened.

[scottbergerphoto]

That's right. The camera displays the embedded JPEG on the review screen, and that JPEG does have the white balance setting applied. The RAW file contains the white balance information from the camera, but it is not applied to the RAW file. Rather, when you view the RAW file in RAW conversion software, the white balance info is applied by the software. However, it's applied in such a way that it is not permanent until you actually convert the RAW file to a JPEG or another format.

I believe that is correct for some Canon digital cameras, but not all. On some models, there is no embedded JPEG in the RAW file.

---Bob Gross---
The 1D Mark II can shoot raw files without an embedded JPEG. I believe the Jpeg is recorded as separate file on the Mark II.
Regards,
Scott

[Longwatcher]

Just a pont of technicality.
RAW images are compressed, just they are compressed in a mathematically lossless format, so that no information is lost from the image (or at least it is not supposed to).

For example: Jpegs use a 8x8 block of pixels and then through some magic I can't remember right now, turns all 64 pixels into one or two numbers, which when put back together don't always look the same as when they started (if you pay close attention to a badly compressed JPEG like at 400% view, you can sometimes see the 8x8 blocks as distinct blocks.

In an example of lossless compression: if the compression engine (software) sees a row of X number of the same value, such as 8 pixels in a row with the same intensity value (say 57), then instead of going 5757575757575757, it goes 57x8 thus 4 bytes instead of 16 bytes or a 4:1 compression. This is why shots with lots of a solid field (such as sky or a plain white wall) will be a smaller file then say pictures of a grassy field or dense wooded area in the background.

I think I got it right (must stop answering questions just before I sign off and go to bed.

[ssim]

The 1D Mark II can shoot raw files without an embedded JPEG. I believe the Jpeg is recorded as separate file on the Mark II.
Regards,
Scott

That is correct. You can turn off th JPG format all together which is what I have done and do nothing except RAW now.

[robertwgross]

That is correct. You can turn off th JPG format all together which is what I have done and do nothing except RAW now.

However, some Canons do not have any RAW+JPEG option at all.

---Bob Gross---

[hmhm]

However, some Canons do not have any RAW+JPEG option at all.

What you see on the LCD preview is a "thumbnail", not an embedded jpeg. Not all Canon DSLRs have embedded jpegs, but I think they all have a notion of a "thumbnail" (look for a .THM file on the flash), which is a super-low-resolution rendition of the image.
-harry

[timmyquest]

*pokes head in*

Everytime i see this thread i come in expecting WWF photos

*looks around and leaves*

[gcogger]

The only actual processing that is done to a Raw file is a proprietary amount of sharpening, amount known olny to Canon, that occurs even with sharpening turned off in the camera.
Source: Chuck Westphall, Director of Technical Information, Canon, USA
Regards,
Scott

How do you turn the sharpening 'off' in the camera? My 300D only has settings of -2 to +2.
Breezebrowser, in addition to these settings, has a 'No sharpening' option that is less sharp than -2. I wonder if Chuck was talking about this - even if you process at -2 sharpening, there is still some sharpening applied. Perhaps the Breezebrowser 'No sharpening' option really IS no sharpening?

[CyberDyneSystems]

The 1D Mark II can shoot raw files without an embedded JPEG. I believe the Jpeg is recorded as separate file on the Mark II.
Regards,
Scott

Actually .. the 1D MkII RAW files DO have a 1/2 size jpeg "embedded" as well.

Do not confuse this 1/2 size embedded joeg with the jpeg you can chose to have alongside the RAW file at the resolution and compression of your choosing..

In addition there IS an embeded 1/2 res jpeg.. whether you wnat it or not. No clue as to what Canon intends it to be used for.. it is not documented or extractable by Canon software.. (Breezebrowser will extract it.. even older versions of Breeze did before the versions that actually suppoorted MkII raw files)

So given that we know the MkII file has this mysterious undocumented embedded jpeg as PART of the RAW file itself.. is it not safe to hypothesize that other Canon RAW files may very well have one as well?

Go figure...

Anyways.. none of this really pertains to the original question. :roll:

[scottbergerphoto]

The only actual processing that is done to a Raw file is a proprietary amount of sharpening, amount known olny to Canon, that occurs even with sharpening turned off in the camera.
Source: Chuck Westphall, Director of Technical Information, Canon, USA
Regards,
Scott

How do you turn the sharpening 'off' in the camera? My 300D only has settings of -2 to +2.
Breezebrowser, in addition to these settings, has a 'No sharpening' option that is less sharp than -2. I wonder if Chuck was talking about this - even if you process at -2 sharpening, there is still some sharpening applied. Perhaps the Breezebrowser 'No sharpening' option really IS no sharpening?
Even with the camera set to -2 or whatever, there is still some sharpening applied to the Raw file that cannot be turned off.
Regards,
Scott

[gcogger]

The only actual processing that is done to a Raw file is a proprietary amount of sharpening, amount known olny to Canon, that occurs even with sharpening turned off in the camera.
Source: Chuck Westphall, Director of Technical Information, Canon, USA
Regards,
Scott

How do you turn the sharpening 'off' in the camera? My 300D only has settings of -2 to +2.
Breezebrowser, in addition to these settings, has a 'No sharpening' option that is less sharp than -2. I wonder if Chuck was talking about this - even if you process at -2 sharpening, there is still some sharpening applied. Perhaps the Breezebrowser 'No sharpening' option really IS no sharpening?
Even with the camera set to -2 or whatever, there is still some sharpening applied to the Raw file that cannot be turned off.
Regards,
Scott

Umm... I think that's what I was trying to say :o
My question about turning sharpening off in the camera was attempting to clarify your quote about '...even with sharpening turned off in the camera'. If you're shooting in RAW, it makes no difference what you set the sharpening to in-camera.
If using -2 on conversion results in some sharpening, then surely that's a property of the converter, not the RAW file itself. The fact that Breezebrowser offers 'No sharpening' as a less sharp option than -2 suggests to me that the sharpening happens in the software not the camera, and that the RAW file really IS unprocessed.
Which, of course, is what this thread is all about :)

[robertwgross]

What you see on the LCD preview is a "thumbnail", not an embedded jpeg. Not all Canon DSLRs have embedded jpegs, but I think they all have a notion of a "thumbnail" (look for a .THM file on the flash), which is a super-low-resolution rendition of the image.
-harry

Yes, but the discussion is not about the accuracy of the thumbnail image. Nor did anybody claim that the LCD showed an embedded JPEG.

---Bob Gross---

[toddb]

Raw is so cool. I remember when my last point and shoot digital shot uncompressed tiffs....they were huge because they were not compressed. To me, Raw is worth it just for the lossless compression alone and being able to maintain the full 12bit per chanel tonal range instead of being downed to 8bit. (4096 down to 256)

For me Raw will be my number one choice because I think the RAW converters will keep getting better and better. So maybe 2 years from now, with Photoshop 10, I can get an even better image out of my current raw files.

[scottbergerphoto]

The only actual processing that is done to a Raw file is a proprietary amount of sharpening, amount known olny to Canon, that occurs even with sharpening turned off in the camera.
Source: Chuck Westphall, Director of Technical Information, Canon, USA
Regards,
Scott

How do you turn the sharpening 'off' in the camera? My 300D only has settings of -2 to +2.
Breezebrowser, in addition to these settings, has a 'No sharpening' option that is less sharp than -2. I wonder if Chuck was talking about this - even if you process at -2 sharpening, there is still some sharpening applied. Perhaps the Breezebrowser 'No sharpening' option really IS no sharpening?
Even with the camera set to -2 or whatever, there is still some sharpening applied to the Raw file that cannot be turned off.
Regards,
Scott

Umm... I think that's what I was trying to say :o
My question about turning sharpening off in the camera was attempting to clarify your quote about '...even with sharpening turned off in the camera'. If you're shooting in RAW, it makes no difference what you set the sharpening to in-camera.
If using -2 on conversion results in some sharpening, then surely that's a property of the converter, not the RAW file itself. The fact that Breezebrowser offers 'No sharpening' as a less sharp option than -2 suggests to me that the sharpening happens in the software not the camera, and that the RAW file really IS unprocessed.
Which, of course, is what this thread is all about :)
No, you are missing the point. Canon does not explain how it is done or how much is done, but there is some sharpening done in the camera to all Raw files regardless of conversion software or camera settings. It's just in there and there is nothing you can do about it. So even if you don't add sharpening anywhere in the workflow, some exists.
Regards,
Scott

[robertwgross]

No, you are missing the point. Canon does not explain how it is done or how much is done, but there is some sharpening done in the camera to all Raw files regardless of conversion software or camera settings. It's just in there and there is nothing you can do about it. So even if you don't add sharpening anywhere in the workflow, some exists.
Regards,
Scott

Scott, what good does this apparent fact do us? We can't get a hold of it. We can't change it. We can't see it. It's just there.

I don't see how it can affect me differently, one way or the other. Does it affect you?

I suspect that we all do a moderate amount of sharpening to our files before they are finished. OK.

---Bob Gross---

[gcogger]

No, you are missing the point. Canon does not explain how it is done or how much is done, but there is some sharpening done in the camera to all Raw files regardless of conversion software or camera settings. It's just in there and there is nothing you can do about it. So even if you don't add sharpening anywhere in the workflow, some exists.
Regards,
Scott

Sorry Scott, but I am certainly NOT missing the point - just wondering whether Chuck really meant that the sharpening was applied in-camera, or he was assuming processing in Canon FVU. From my original post:

Breezebrowser, in addition to these settings, has a 'No sharpening' option that is less sharp than -2. I wonder if Chuck was talking about this - even if you process at -2 sharpening, there is still some sharpening applied. Perhaps the Breezebrowser 'No sharpening' option really IS no sharpening?

I was also trying to clarify why you were talking about 'turning off sharpening' in-camera, and using -2 sharpening in-camera, since these settings have no effect on the RAW file.

[scottbergerphoto]

No, you are missing the point. Canon does not explain how it is done or how much is done, but there is some sharpening done in the camera to all Raw files regardless of conversion software or camera settings. It's just in there and there is nothing you can do about it. So even if you don't add sharpening anywhere in the workflow, some exists.
Regards,
Scott

Scott, what good does this apparent fact do us? We can't get a hold of it. We can't change it. We can't see it. It's just there.

I don't see how it can affect me differently, one way or the other. Does it affect you?

I suspect that we all do a moderate amount of sharpening to our files before they are finished. OK.

---Bob Gross---
Bob,
Sometimes knowledge is good just for knowledge sake. I think it is interesting that Canon Raw files have some processing done to them even though I can't change it. I believe it's called "intellectual curiosity".
You curmudgeon you!
Regards,
Scott

[Longwatcher]

Scott, what good does this apparent fact do us? We can't get a hold of it. We can't change it. We can't see it. It's just there.

I don't see how it can affect me differently, one way or the other. Does it affect you?

I suspect that we all do a moderate amount of sharpening to our files before they are finished. OK.

---Bob Gross---

I have to add that it does affect us. As an example, those crappy shots you get out of a Kodak SLR/c (and 14N) are a result of even less processing done to the images then Canon does to them. No digital shot can be taken without some processing. Just turning the detections into something useable requires at least minimal processing. Mainly making sure that all of the pixels are in thir correct locations.

Now on to the point; because Canon does some in camera processing, noise reduction (color, moire, intesity at least) and a small bit of in-camera sharpening, the pictures you get out are very good to begin with. Where as Kodak opted to let the photographer have even more control over the RAW file versus final product and they pay for it by bad press. This lack of in camera processing affects the jpeg images the Kodak produces as well, since they apparently decided that (I am guessing here) that the majority of their photographers would be using RAW images and only use jpegs for a quick look.

So the fact that Canon does do in camera processing does affect which camera you and I buy. That can make a huge difference.

After you have the camera though it is time to stop worrying until you want your next camera.

Just my opinion, feel free to ignore.

[hmhm]

Yes, but the discussion is not about the accuracy of the thumbnail image.

The originator of this thread said:
"Perhaps it really isn't altered, and what I'm seeing on the back of the camera when I change the white balance are JPGs created to give review images. That is, you're not really looking at the RAW file on the review screen?"

-harry

[theoldmoose]

In an attempt to be most correct about what Chuck Westfall said, it is true that most Canon cameras that produce RAWs do some in-camera sharpening, and that it cannot be turned off or circumvented by computer software after the fact (by selecting a 'no-sharpening' option).

HOWEVER, the single exception to all this are the small class of cameras (the 1ds is one of them) that can produce a raw TIFF file. These files (and these files only) have had *no* sharpening applied to them.

If you search the robgalbraith forum archives, you will dig up Chuck's posting on this subject. He was quite clear about this.

[robertwgross]

Hey, I have an idea.

Why don't we drag Chuck Westfall in and sit him down in front of some hot lights? Then we can get some straight answers from the horse's mouth.

---Bob Gross---

[Scottes]

Is Chuck talking about the sharpening required by the Bayer filter on the sensor used for anti-aliasing? This sharpening is really needed, and from what I understand it specialized to combat the effects of the Bayer filter.

If it's this, then you can't change it, nor would you want to do so.


I do remember reading a thread with Chuck saying something along the lines of "some sharpening is done that you can't disable", but I don't remember why or how. And I don't feel like researching it.

[skid00skid00]

Per Chuck Westfal at another forum, the 10D sharpens RAW files in-camera, even when sharpening is set to off/lowest.

The 1Ds does NOT sharpen RAW. (I don't know about the 1D.)