Yarc : A RAW converter for G1, too

Uhhh, a few questions

Hi All,
Well my last post somehow didn't show up and so I will try this again...hopefully it won't be a repeat with the original post showing up sometime later!
Just a few questions here...
What's the use of converting Raw to tiff without the ability to adjust the white point first? I do like Yarc as it has many fine abilities, but isn't white point selection the main reason to shoot in Raw? Or...am I missing something here?
nahau

Nahau,
resolution is another reason. Try photographing small detailed objects, such as hair, in both JPEG S and CRW, enlarge a crop 400% or more, and compare... visible difference.
I now use Yarc to generate the index (-jw50), and I still use the Twain import plug-in in Photoshop.

nadim

Hi Nadim,
Yes I realize that Raw offers much more resolution, but I was just curious since no one else seemed to be concerned with this aspect.
Is the Yarc then just for indexing/EXIF/viewing.​.. and normal Raw conversion still through Twain/Zoombrowser/RawC​onverter? You know how picky I am about white point! LOL!! I guess this is what must be done then! By the way, I purchased Photoshop 5.5 from a friend who upgraded to ver 6.0 (higher??). He gave it to me for a really good price. . I don't know why he didn't want to just use the upgrade package...but I'm not complaining! Now whenever you speak "Photoshop", I can "try" to follow along!
nahau

Yarc1.4.8b-FCF Beware the FCF

I have discovered that the new (Version 3.6.1) Canon DLLs made available recently, though much faster in conversion time, have the False Color Filter option OFF by default. For myself, and I suspect most Yarc users, this is NOT a good thing. This release reverts back to version 3.5 as distributed with Yarc1.4.5b and earlier. I will continue to work on finding out how to make the FCF optional.

Note: Files converted with
- ZoomBrowser will have the False Color Filter ON.
- Image Converter will have it OFF
- Twain driver may select FCF either ON or OFF (check box)

You can download the latest version of Yarc at:
http://www.roava.net/~​henderbc/yarc.htm

Bruce

nadim
--------

I'm glad Yarc has simplified your life - I know it has mine. As for the aspect ratio for G1 and Pro90, you're right - it's screwed up - it is optimized for the D30 which has a 2:3 aspect ratio. Fixing this is on my list.

> Anyhow, thanks a lot for sharing your program for
> free with the Canon user community. Great utility,
> nadim

Although Yarc is free, your Yarc licence agreement has an alternative way to say thanks.

Andrei
--------
> Yarc can't generate JPEG ( -jw) from CRW, shot
> VERTICALLY + SATURATION=LOW

Please e-mail me this CRW+THM - I will investigate.

> what are your plans about GUI ? It looks like you are
> a system programmer, aren't you

I have done a lot of low-level programming over the years, but in recent years have been doing mostly Windows application software. As to why Yarc has a command-line rather than a GUI interface, there are a couple of good reasons:

First, I believe it's very important to keep focused on function rather than form in the early stages of software development. In the case of Yarc, which is still very much a work in progress, working on a nice GUI would distract me from thinking about function. ZoomBrowser is, in my opinion, the quintessential example of what you get when you think about form and not function.

Second, the raison-d-etre for Yarc was to have a way to do batched operations so I could overlap CRW conversion with my PS and other work. This goal lends itself very nicely to a command-line approach. I am not out to design the next ZoomBrowser, Bibble, ACDSee, Photoshop, etc. In fact, I sincerely hope that some commercial software will materialize that has my needs in mind. If this doesn't happen, I will probably wrap Yarc in a GUI so I don't have to keep remembering all those $^&@^%#@*&^ switches.

Nahau
--------
My reason for shooting RAW is quite basic. I think about my image processing in two stages. Stage 1 involves getting the image cropped and rotated correctly, and getting exposure and color balance right. Stage 2 is concerned with getting all the fine details right. I have come to learn that it is essential for me to do stage 1 in 16-bit mode.

The main reason for this is that it gives me have MUCH more latitude in making aggressive adjustments. If I could nail my exposure and color balance every time, then JPG is perfectly acceptable to me. The cardinal rule of digital photography seems to be: If you must expose wrong, make sure it is under. Once you overexpose and blow out some highlights, the image is ruined.

I also find that the noise situation is improved by shooting RAW. JPG by design doesn't do a very good job of compressing fast edges and impulse noise. When you shoot high ISO, a bad thing (noise) gets made worse by JPG (artifacts).

Now, as to the white point, see my reply to Andrei. I think PhotoShop is a marvelous piece of software. I can think of no better place for me to adjust the white point. To have that done during the CRW conversion buys me nothing. In fact, to have it done in the camera buys me nothing unless the D30 did it in analog hardware prior to its D/A conversion, which I am at least 99.9% sure it does NOT. Given this fact, I would much rather defer making white point decisions until PS where I have a lot of great tools to deal with it.

hope this helps, Bruce

An afterthought (to G1/Pro90 users)

It seems that a lot of readers of this forum are G1 (and Pro90?) users, so I should point out that my previous remarks about RAW mode were base on my D30 experience. With the 12-bit A/D converters in the D30, the argument for 16-bit mode is more convincing.

In the G1, you are starting with 8-bit data. So, I am interested in hearing your thoughts on RAW shooting in a G1.

I should also add that I believe the False Color Filtering issue applies only to the D30. If I am wrong about this, please let me know.

Bruce

Hi Bruce

Very nice program you have created. I just got it today. I am using it with my G1.

I have placed a shortcut on my desktop. Then I just grab my shots (from SanDisk and 340 Mb MicroDrive) and place them over the icon.

But is there a limit in the number of pic`s that you can convert (drag over to the icon) in one job.

I had an error when I tryed to convert a whole dir (from the harddisk). Up to 8 pic`s everything worked fine.

It would be nice if Yarc could arrange the outputs in different formats-dirs. Tiff`s in one, jpg in another and so on.

Is it possible ?

Best regards
Sherman

Sherman,

You didn't say what OS you are using. Drag'n'Drop behavior isn't consistent across all Windows flavors. Did you select multiple files and then drag them? If so, try dragging the folder they're in instead. This will accomplish the same thing. Let me know if this doesn't do the trick.

Regarding the output folders, yes it is possible. I'll put your suggestion on my wish list.

Bruce

Hi Bruce,
First off, thanks for this fine program and your insight on Raw files. It seems though that the white point adjustment is quite important to me. What is pleasant to my eyes seems to be unattainable with the current ICC profiles resident in Photoshop or PSP7. I guess I am stuck with the Rawconverter for now...maybe Canon will make it faster one day (unlikely??). I have since hacked my Win98 registry to add right-click functionality for using your program with switches and it works very well and fast thus allowing me to generate only the jpegs and EXIF files in the current Raw folder...I will generate tifs through Rawconverter as I need them. Being able to "see" the Raw files before conversion to tif though is a definite plus!
There seems to be one problem though and I don't know if it's my Twain driver (ver 3.6.1) or Yarc, or something else. When I use the Twain to try to import Raw images into an application from the Raw folder that has been "Yarced" (jpegs, EXIF, crw, thm, tif, and index), the Twain hangs on the jpegs. It doesn't totally hang my system, just the import window. If I remove just the jpegs from the folder, and go through Twain import again, it works fine. Maybe since the Twain is trying to also import the jpegs (and they have the same name), its messing up somehow. Have you (or anyone else) experienced this at all? BTW, I tried this with/without the registry hacks and the same thing happens. Thanks!
nahau

nahau,

It sounds to me like you would like to get all your corrections done prior to Photoshop, or without Photoshop altogether. If this is the case, you might be happier with a Bibble solution.

Regarding the Twain driver: I have seen exactly the same behavior. I suspect it also afflicts ZoomBrowser but I haven't tried it recently.

The Canon Dlls implement their own flavor of directory management, and seem to have been written with a very narrow view of what kinds of other files people might have in the directory that contains their CRW files. I'm not sure, but I think Twain becomes confused with types of files other than JPG, or subdirectories under where you keep your CRWs.

regards, Bruce

Yarc1.4.9b is a minor release which includes:

-fixes aspect ratio of JPG preview web page for G1 and Pro90 cameras

-when processing a directory of CRW files, files are now processed in proper alphabetical order.

-when extracting from files with the read-only attribute set, Yarc would do the extraction the first time, but would then fail if run again on the same file. This is now fixed.

For D30 Users: NOTE: TIFF extraction with this release is done with the False Color Filter turned ON. See the Readme.htm if you don't know what this means.

You can get the latest version of Yarc at:
http://www.roava.net/~​henderbc/yarc.htm

enjoy, Bruce

Hi Bruce,

I haven't tried Yarc yet (just downloaded) but by reading the readme file on your website and the users' comments here, it sounds like a great utility i should try out......I'm a big fan of fast efficient command line operations, being a Linux fan and all! I was thinking of wrapping your program with a IIS ASP based web app for my own personal use to provide the GUI needed without the hassle of dealing with Win32 GUI programming.......

However, I have the same concerns as nahau - I would like to perform image corrections in during RAW conversion before using Photoshop to further correct the images. Specifically, I need the ability to pick a custom white point. It's the main reason that I shoot in RAW.

I have 2 suggestions for the program in this aspect:

1) Show the first picture ala RawConverter, let the user choose his parameters, and let yarc somehow save those custom settings and apply them to the rest of the images in batch conversion. This would at least allow us to choose a custom white point for ALL images shot in the same settings.....
(Of course I realise saving a custom white point may not be easy, if possible....)

2) Alternatively, allow yarc to accept command line parameters that tells the program which RAW parameters need to be changed, for example, call yarc with parameters to tell it to set white point to "Cloudy", "Low Sharpness", "Low Contrast" etc. We will not be able to set a custom white point for this, but at least we would be able to change the RAW parameters to those fixed definable ones......

finally thanks for your hard work...will try out the program tonight!

Bruce, is it possible to add options to extract JPEG files only ?

To extract the JPEG in requested size (800x600 for examle) ?

Bruce Henderson wrote:
Yarc1.4.9b is a minor release which includes:

-fixes aspect ratio of JPG preview web page for G1 and Pro90 cameras

-when processing a directory of CRW files, files are now processed in proper alphabetical order.

-when extracting from files with the read-only attribute set, Yarc would do the extraction the first time, but would then fail if run again on the same file. This is now fixed.

For D30 Users: NOTE: TIFF extraction with this release is done with the False Color Filter turned ON. See the Readme.htm if you don't know what this means.

You can get the latest version of Yarc at:
http://www.roava.net/~​henderbc/yarc.htm

enjoy, Bruce

reddawn,

>I would like to perform image corrections in during
>RAW conversion before using Photoshop to further
>correct the images. Specifically, I need the ability to
>pick a custom white point. It's the main reason that I
>shoot in RAW.

When you (and nahau) use the term 'white point', I'm not sure whether you mean 'choose a custom white balance correction', or 'choose which luminance level corresponds to levels 255,255,255 (white) in Photoshop'. I will assume that you mean the former. If not, forgive the misunderstanding.

Yarc is of course tuned to MY process - I wrote it to simplify my life. So I if you understand my process, though you may not be convinced Yarc works for you, I hope you'll at least understand why it is the way it is.

First of all, I shoot RAW, not so I can use primitive tools such as ZoomBrowser or Raw Image Converter to do my white balance correction, but rather so I can convey my precious pixels with all the dynamic range that was captured, and with as little damage as possible into Photoshop where I find professional calibre tools to deal with color correction issues. For example, I have found that the correct settings for color balance and those for correct levels can interact. For this reason, I don't like to commit to fine color balance adjustments until I at least have the levels in the ballpark.

Secondly, I take it as a given that for any given image, none of Canon's (or anyone else's) presets for color balance are really correct - some are just less incorrect than others. There are an infinity of lighting situations, and each is a little different than any other. I therefore seek only to shoot with a color balance setting that is 'in the ballpark'. As long as I can get this image into PS in 16-bit form, I'm happy.

For these reasons, I shoot almost all my naturally lit subjects with the 'Sunny' camera setting. If I am shooting on a really steel-gray day, I may pick 'Cloudy' to add a little warmth, but I've found that the difference between these two settings is very subtle, and I'd really rather make that decision later in PS where I'm better equipped to do so.

For flash, I also use Sunny. The color temperature of my 550Ex is close enough to daylight so it's not worth the bother of switching.

For artificial light, I just pick Tungsten or Fluorescent, as appropriate. For mixed lighting, I just try to pick whatever I think dominates, unless the shot is really important, in which case I'll do a custom WB with a gray card or a white wall.

If I've made the correct choice on the camera, the default behavior of Yarc is correct - it uses the as-shot settings to make the TIFF. But sometimes in difficult lighting situations, or if I forget to check the WB camera setting, I get the camera setting wrong. In this case, Yarc lets me override the as-shot WB setting with the -p switch.

So, how to decide when to do an override? That's what the preview JPGs are for. The JPG Yarc extracts from the CRW file is of low quality relative to what's really in the CRW file, but is of immensely more use than the THM files. Given that I care enough to undergo the bother of shooting RAW, this JPG is of no permanent value to me. It is however a highly effective way to screen my shots. I can easily tell enough about sharpness, exposure, and of course composition to know whether to trash the shot without further ado. I can also catch WB mistakes I might have made with the camera setting.

Yarc is designed to produce all the JPGs before producing any of the TIFFs (when you use the -jt switch you get both). This is done so I can preview them and make informed decisions about my shots. After the JPGs are made, Yarc starts grinding away at the TIFF files, but it does so as a background task (below the priority of my desktop apps - ZoomBrowser and Raw Image Converter both bring my computer to it's proverbial knees, so trying to do anything else in parallel is hopeless).

If I find I made a mistake on a couple of shots, I just note them and go back later and rerun Yarc on them with the appropriate -p switch. If I realize that the whole batch is bad (ie: Sunny WB in incandescent lighting) I just Ctrl-C to quit Yarc and rerun with a different -pw setting. For situations like these, Yarc lets me enter a range of files (ie: Yarc -t CRW_1234..1244) to confine it to only the range that was wrong.

After the first TIFF is cranked out, I can go to work on it with PS immediately. This is in contrast to the Twain module, which ties up my PS until all images are imported. I find that it ALWAYS takes me longer to complete my PS work on an image than Yarc takes to make a TIFF. This produces the happy consequence that my TIFF conversions are free. Because they're pipelined with my PS editing, and done at a background priority, the conversion time becomes completely transparent to my process.

With the foregoing as a lengthy introduction, let me address your specific suggestions.

>1) Show the first picture ala RawConverter, let the
>user choose his parameters...

This is exactly what the JPG previews are for - only you get to consider the whole shoot - not just the first picture of a series. You can of course choose your parameters with the -p options.

>2) Alternatively, allow yarc to accept command line
>parameters

See >1)

I hope I haven't bored you with this rather windy explanation.

regards, Bruce

PS: Your "IIS ASP based web wrapper" sounds interesting - please let me know what you end up with.

Andrei,

>is it possible to add options to extract JPEG files only ?
>To extract the JPEG in requested size...

Yes - it's possible, however, to do so would a) be a bunch of work for me, and b) not be consistent with my goals and my process.

To undergo the hassle of dealing with CRW files, one has to care a lot about one's images. I'm willing to commit to a (compressed) JPG only after ALL corrections have been made. I NEVER edit a JPG that originated with one of my CRW files.

What exactly would you do with such a JPG if you had it?

regards, Bruce