Color space and Lightroom (be kind! :-) )

All

I need some help and advice from this extremely knowledgeable board!

I've only recently started to take my photography a little more seriously, coming from years of P&S cameras and taking simple snapshots. Pls be kind to a noob....I've been reading and trying to learn so much about color space, ICC profiles, etc that my brain's gone a little numb!

I now have a Canon 50D with Tokina 11-16MM f/2.8, Canon 50MM f/1.8 MkII, Tokina 100MM macro f/2.8 lenses, and an HP Photosmart Pro B8850 wide-format inkjet.

I shoot only in RAW, and have been tweaking in DPP. I have been editing in DPP in the Wide Gamut space, and copying the physical image files to another directory and converting to sRGB, saving them, then batch processing to JPG with ICC profile (sRGB) embedded.

I want to ensure that my image files are stored permanently in the HIGHEST possible quality, so that future devices (monitor/printer/web browsers) can take advantage of the quality of the pics in years to come, as they expand/enhance their color space range. (excuse any incorrect terminology, but I hope you understand my view here....)

I'm quite happy to make copies of the high quality images for web and printing in sRGB, as necessary.

So this is why I need your advice!

I'm looking at software options for image management (repository/store) and a more comprehensive image editing package, but before I purchase anything, I need to ask for someone who really knows and understands color spaces if my understanding is correct, as below.

1. As I shoot only in RAW, and I don't print directly from the camera, my understanding is that the in-camera Color space setting is irrelevant to me. (I have it set on Adobe Raw, FWIW). Is this a true statement?

2. I am looking at Lightroom V3 for my image storage and basic editing. Is it true that Lightroom 3 imports and converts RAW images into a ProPhoto RGB color space which is a "wide" color space and the only manipulation with color space conversions is then when exporting the pics for web and print?

3. Would there be any potential color quality reductions/differences during import into Lightroom from DPP, if I ....

-imported images which had been edited and saved in DPP as "wide gamut" or "Adobe RGB" files, assuming the full color space was used in "wide gamut" editing, compared to
- importing images which had been converted in DPP from "wide gamut" above to sRGB and saved as such in DPP prior to importing into Lightroom?

4. As I currently have 3 physical versions for each image (DPP "wide gamut" files for my permanent high quality storage and edits, plus the "sRGB" versions of the same files, plus the converted JPG files from the sRGB versions), when I import them into Lightroom, I will get 3 different versions of the files in the catalog. I can easily separate the JPG versions out, but would the "wide gamut" and sRGB RAW files be any different in Lightroom, and how would I be able to distinguish between them in Lightroom?

5. Or would they be identical and I could delete one of the 2 versions in Lightroom?


Sorry for the very specific questions, folks, and please set me straight if my understanding/assumpti​ons (or "workflows" are wrong/bad).

I appreciate any help or advice you can provide me!

OB

Well, let's see if I'm straight on this -- you have Raw files that you work on in the Wide Gamut DPP color space, and then you copied the Raw files into another directory and work on them in the sRGB color space -- correct? So they are both Raw files and then when you want to convert to jpegs you do so from the ones you work on in sRGB?

I want to make sure we are clear on this...

Offhand, I don't thing you can import more than one copy of the Raw file into Lightroom unless you rename it. The edits done in DPP don't show up in Lightroom, Lightroom will ignore any color space "tag" DPP applies to a Raw file (although it will recognize a jpeg converted and saved with the sRGB space).

In fact, I'm not sure how your desktop printer will handle a print job using the DPP Wide Gamut color space -- you'll want to play a bit before doing any big critical jobs! Using Adobe RGB should work fine if you need the color range of aRGB.

This is kind of a complicated scenario -- maybe someone who has some experience working with the Wide Gamut space can chime in here...

tonylong wrote in post #11836022
Well, let's see if I'm straight on this -- you have Raw files that you work on in the Wide Gamut DPP color space, and then you copied the Raw files into another directory and work on them in the sRGB color space -- correct? So they are both Raw files and then when you want to convert to jpegs you do so from the ones you work on in sRGB?

Correct. I've been using only DPP until now, and worked this way (interim step of saving the wide gamut RAW files as sRGB RAW files PRIOR to batch converting them to JPG, as I was never sure if it worked properly if I switched the color space AFTER rendering to JPG directly from Wide Gamut, in DPP.

tonylong wrote in post #11836022
Offhand, I don't thing you can import more than one copy of the Raw file into Lightroom unless you rename it.

You can. It seems to tag a "-2" or similar suffix ontot he end of the duplicate file names it finds. I did not want to use the "Don't copy duplicates" option in the Import, as I wasn't sure precisely WHICH duplicate it would bring in....the JPG or the sRGB or the Wide gamut versions.

tonylong wrote in post #11836022
The edits done in DPP don't show up in Lightroom, Lightroom will ignore any color space "tag" DPP applies to a Raw file (although it will recognize a jpeg converted and saved with the sRGB space).

In fact, I'm not sure how your desktop printer will handle a print job using the DPP Wide Gamut color space -- you'll want to play a bit before doing any big critical jobs! Using Adobe RGB should work fine if you need the color range of aRGB.

This is kind of a complicated scenario -- maybe someone who has some experience working with the Wide Gamut space can chime in here...

Thanks. I'm sure I've probably over-complicated this stuff, but didn't want to "lose" any color space in my images due to converting them to a smaller space and unintentionally losing the full color attributes of the pics I've already shot.

Old Baldy wrote in post #11836143
Thanks. I'm sure I've probably over-complicated this stuff, but didn't want to "lose" any color space in my images due to converting them to a smaller space and unintentionally losing the full color attributes of the pics I've already shot.

OK, the Raw files have no embedded color space so they can't "lose" anything -- once you convert to an sRGB jpeg then that is it as far as color range. Converting back to a wider gamut won't gain you colors back. But the Raw file is intact. So, what you do in DPP with a Raw file can be saved -- DPP writes into the proprietary file metadata, not the Raw data itself, and you can mess with the file in Lightroom (Lightroom does not write to the file) and go back to DPP.

So you know, Lightroom does operate in its own internal color space -- a derivative of ProPhotoRGB ("MelissaRGB"). It does not apply a color space until you export an image to tiff/psd/jpeg, or use the Edit in Photoshop function.

So, in other words, as long as you are working with Raw, DPP and Lightroom are "invisible" to each other.

OK, this is VERY helpful to me, Tony!

As I move away from DPP and start to use Lightroom as my digital store (possibly with Photoshop for some edits), and start to export/print from Lightroom, I have no need then for anything other than the original RAW file, and so I can delete the existing JPG files and either one of the two RAW file versions that I moved into my trial version of Lightroom 3.

This is exactly what I wanted to know...thanks!

That's my arrangement. I work off the Raw files -- and in both DPP and LR you can have different "versions" of your image without creating a copy. With DPP you can use the Edit/Save recipe to file command so you can load a version at will, make changes, etc.

Lightroom has a nifty utility called Virtual Copies that is quick and easy (I use it all the time). At any point you can right-click on an image (or, if selected, multiple images) and select Create Virtual copy/copies. A thumbnail appears and can be opened and developed right alongside the original, but it is not a file, just metadata that you can use to prepare a whole version and you can import at will. I do that a lot if I'm preparing a cropped version, a B&W version, etc.

I think you have a basic misunderstanding of how DPP and other converters work. The working space setting in DPP, whether in Preferences or from the menu bar, refers to three things; the space in which the output RGB file (jpg or tif) will be written, the space into which histogram data is converted and (apparently)the space into which display data is converted before its conversion to the monitor space. What it does not refer to is the internal working space in which DPP edits the RAW data. Canon is not as forthcoming as Adobe about the RAW workflow, so we don't know what that internal space is, but it surely is not smaller than WideGamut RGB and may in fact be larger than either WideGamut RGB or ProPhoto RGB (which are similar in size) because the native capture gamut of modern cameras is larger than those spaces. That same space is always used. There is no such thing as a "WideGamut RAW" or a "sRGB RAW", because no matter what you do DPP will never change the RAW data. The "Save" operation in DPP saves only instructions from DPP to itself, instructions that can be changed at any time by moving a slider. Therefore, it is entirely unnecessary to copy the CR2 file. Since the conversion to a matrix based working space takes place only at the time of saving the RGB file, the same RAW file can one day be converted to a 16 bit WideGamut tif and the next day to a sRGB jpg. As long as you preserve the original CR2 you have all the data captured by the camera (or at least all that Canon gives you).

As regards Lightroom, we do know that the RAW data is translated into a space called Melissa RGB which has the same gamut as ProPhoto RGB but a gamma of 1.0. Again, this space is a hard-wired part of the internal workflow and cannot be changed. And again it should be emphasized that the RAW file is itself unaltered, it is only data read from the file and entered into memory that is converted.

As Tony said, it makes no sense to open a CR2 first in DPP, edit it and then open it in LR. The only thing that DPP will do to the CR2 is to insert editing instructions which are incomprehensible to LR. Moreover, doing that would imply that DPP editing is necessary as a first step because it is superior to LR editing, which I refute (although to be fair I should note that some people do prefer it).

The bottom line - to preserve all the color data just hang on to the RAW file. However, you should also realize that any use of this data to display it either electronically or on paper will inevitably involve a reduction of the data.

Edit: I type slowly and Tony has already answered your questions, but I'll leave this up.

tzalman wrote in post #11836483
I think you have a basic misunderstanding of how DPP and other converters work. The working space setting in DPP, whether in Preferences or from the menu bar, refers to three things; the space in which the output RGB file (jpg or tif) will be written, the space into which histogram data is converted and (apparently)the space into which display data is converted before its conversion to the monitor space. What it does not refer to is the internal working space in which DPP edits the RAW data. Canon is not as forthcoming as Adobe about the RAW workflow, so we don't know what that internal space is, but it surely is not smaller than WideGamut RGB and may in fact be larger than either WideGamut RGB or ProPhoto RGB (which are similar in size) because the native capture gamut of modern cameras is larger than those spaces. That same space is always used. There is no such thing as a "WideGamut RAW" or a "sRGB RAW", because no matter what you do DPP will never change the RAW data. The "Save" operation in DPP saves only instructions from DPP to itself, instructions that can be changed at any time by moving a slider. Therefore, it is entirely unnecessary to copy the CR2 file. Since the conversion to a matrix based working space takes place only at the time of saving the RGB file, the same RAW file can one day be converted to a 16 bit WideGamut tif and the next day to a sRGB jpg. As long as you preserve the original CR2 you have all the data captured by the camera (or at least all that Canon gives you).

As regards Lightroom, we do know that the RAW data is translated into a space called Melissa RGB which has the same gamut as ProPhoto RGB but a gamma of 1.0. Again, this space is a hard-wired part of the internal workflow and cannot be changed. And again it should be emphasized that the RAW file is itself unaltered, it is only data read from the file and entered into memory that is converted.

As Tony said, it makes no sense to open a CR2 first in DPP, edit it and then open it in LR. The only thing that DPP will do to the CR2 is to insert editing instructions which are incomprehensible to LR. Moreover, doing that would imply that DPP editing is necessary as a first step because it is superior to LR editing, which I refute (although to be fair I should note that some people do prefer it).

The bottom line - to preserve all the color data just hang on to the RAW file. However, you should also realize that any use of this data to display it either electronically or on paper will inevitably involve a reduction of the data.

Edit: I type slowly and Tony has already answered your questions, but I'll leave this up.

Thank you for your feedback/advice. I'm starting to get an understanding of this now, from these posts.

One further clarification, if I may.......where you say "As long as you preserve the original CR2 you have all the data captured by the camera", I take it that you mean as long as I have the physical RAW file, whether or not that has been edited in DPP or Lightroom, then I have the full color data available at all time (as both DPP and LR are non-destructive in their edits, and I can always go back to the original file state at any time....right?

BTW, I didn't mean to imply that DPP is superior to LR in any color / edit respect....by first editing in DPP. That's just a historical workflow where I have images already edited in DPP, that I am now intending to bring into LR and effectively stop using DPP for the most part. I do now understand that all edits I made in DPP are effectively "lost" (or rather, ignored) in LR, so if I ever wanted to replicate the changes to the original camera RAW images in LR, I'd have to execute similar edits in LR (assuming I will work solely in Lightroom/Photoshop and ignore DPP in the future.)
Thanks again,

OB

Old Baldy wrote in post #11836562
Thank you for your feedback/advice. I'm starting to get an understanding of this now, from these posts.

One further clarification, if I may.......where you say "As long as you preserve the original CR2 you have all the data captured by the camera", I take it that you mean as long as I have the physical RAW file, whether or not that has been edited in DPP or Lightroom, then I have the full color data available at all time (as both DPP and LR are non-destructive in their edits, and I can always go back to the original file state at any time....right?

Thanks again,

OB

Right. The RAW file is like a roll of film that needs to be developed, but better than that, it's a roll of magic film that always remains in its original form no matter how many times or in how many different ways you "develop" it.

tzalman wrote in post #11836596
Right. The RAW file is like a roll of film that needs to be developed, but better than that, it's a roll of magic film that always remains in its original form no matter how many times or in how many different ways you "develop" it.

Got it.....now I just have to figure out how to remove all the duplicate RAW and JPG files that I imported (moved) into Lightroom from my three DPP directory structures into 1 LR structure! (Yes, I know that was stupid! )

More LR reading needed......

Do the deleting from within LR. What I do when I want to get rid of a entire folder is to right-click on the folder and select "Show in Explorer" and then delete it in Explorer. Then a second right-click and "Remove from catalog".

My problem is that I consolidated and moved all my many disorganized directories on my hard drive into a single LR structure. I did this to quickly bring in ALL my pics scattered all over the place, thinking that I could easily find all duplicates within LR and remove what I don't need via file filters or something.
So, I now have 20+k images which are roughly three versions of eahc file, thus really have only about 7-8k RAW and original in-camera JPG (before I bought the dslr) images, all in ONE LR file structure. I can't simply delete a single or a few old directory structures, because I effectively consolidated everything into 1 structure.

I'm reading the LR manual, to see if there's any easy way to "find duplicates"....or perhaps consider some plug-in that may do what I need to clean this up.

Well, OB, it looks like you have a job ahead of you!

I'd say take your time -- work out a new workflow based on the discussions here -- a Raw workflow can be very efficient and timesaving as long as you think things through in your process.

Then, over time, you can revisit your previous work, one day at a time. I think you'll find that as you go and learn your way in Raw processing revisiting those older shots can inspire you to take new approaches and create new images!