Important issue for all RAW photographers

[Longwatcher]

Saw this article on luminous-lanscape.com and thought it was very good at stating the same things I believe in as far as eventual problems with the RAW formats and what we as photographer's should do about it.

http://luminous-landscape.com/essays/raw-flaw.shtml

I am of the opinion, this forum site should also support the open raw work group by providing a sticky link on the forum pages, but of course that is not my call to make, just my recommendation so we can educate other photographers to this problem.

http://www.openraw.org/

I have already sent my opinion to Canon on this issue at least once. Recommend anyone who shoots in raw format do the same.

[smittymike19]

Saw this article on luminous-lanscape.com and thought it was very good at stating the same things I believe in as far as eventual problems with the RAW formats and what we as photographer's should do about it.

http://luminous-landscape.com/essays/raw-flaw.shtml

I am of the opinion, this forum site should also support the open raw work group by providing a sticky link on the forum pages, but of course that is not my call to make, just my recommendation so we can educate other photographers to this problem.

http://www.openraw.org/

I have already sent my opinion to Canon on this issue at least once. Recommend anyone who shoots in raw format do the same.


this is nothing but conspiracy theory. the day canon goes out of business ill start sweating. until then im not too worried. its not like they will just drop support of teh raw files one day and noone will be able to help. there are millions of cameras out there, so someone will be able to create a prog that will re encode this to the latest format. Believe it or not i can take an 8 track/cassette/5.25 floppy as stated in article) and i can copy them to my computer in todays format. I can take an 8track and convert to mp3, and 8 tracks havent been made in 20 years!!!!! give it a rest and stop being paranoid. it will happen. raw files will become obsolete. but before it does, there will be programs that can convert to wht or zzz or xxy files, or whatever they call them in the future. when they tell you there is no more raw file format, convert the files. until then, sit back, open an mp3 made from your favorite 8 track and relax.;)

[lostdoggy]

That was rather harsh Mike!!!

[lomond]

That was rather harsh Mike!!!

I agree, it is slightly harsh, but maybe he has a point. :?

[jimsolt]

Harsh or not this view is not far off the mark -- though I think the conspiracy if there is one, is actually on the part of the camera manufacturers. While all that Mike said is basically true, it has become more difficult to do what he claims is possible and that is a problem that will become more rather than less severe. For instance Adobe has put one in the position of buying their new edition CS2 or converting to DNG if you want to shoot RAW in some very popular new cameras. It should in fairness be pointed out that Adobe didn't change the system, the camera manufacturers did.

Yes, there are ways around it, but they are getting more combersome. One way is to abandom Adobe and stick with, say Canon. Unfortunately I don't hear many advocates for Canon's post processing program being better or as good as Photoshop.

There is always a problem like this with new technology. There were 2 competing incompatible systems of color TV. There were 2 competing incompatible systems of consumer video tape.

I'm not defending any view on this issue but my own selfish one. I wish the camera manufacturers would agree on a standard. My experience tells me, "Don't hold your breath."

Jim

[Rob612]

Yes, I agree, Mike has a point. In these last years - especially in the IT business - the OPENwhatever has been a little above the lines, IMHO. RAW files are actually part of the IT world, and it was easily predictable that sooner or later we would have also the OPENraw...

I'll stick with the technology that we have today and go ahead. My most important shots, already fixed, edited etc are all available in huge TIFF format, just to make sure. But I still keep the RAWs.

[Longwatcher]

this is nothing but conspiracy theory. the day canon goes out of business ill start sweating. until then im not too worried. its not like they will just drop support of teh raw files one day and noone will be able to help. there are millions of cameras out there, so someone will be able to create a prog that will re encode this to the latest format. Believe it or not i can take an 8 track/cassette/5.25 floppy as stated in article) and i can copy them to my computer in todays format. I can take an 8track and convert to mp3, and 8 tracks havent been made in 20 years!!!!! give it a rest and stop being paranoid. it will happen. raw files will become obsolete. but before it does, there will be programs that can convert to wht or zzz or xxy files, or whatever they call them in the future. when they tell you there is no more raw file format, convert the files. until then, sit back, open an mp3 made from your favorite 8 track and relax.;)

I have to think you are very naive in your assumptions.

I am happy you still have a functioning 8-track player and a 5.25inch floppy drive available to you. I can still do cassette and vinyl myself, I was given all of my Grandmother's records because I was the only one in the family with a working turntable left. A lot of these records are classics that have no significant market value to be produced on CD, so I need to do it myself if I want to be able to play this music in the future.

However, I can't run several of my favorite DOS based games on either of my computers anymore because of clock-timing problems. I can't without paying someone a LOT of money get someone to convert a laser disk to DVD or CD format. My old 250MB zip drives are pretty much useless unless I can snag someone with an old drive that still works, luckily I convert data to CD before my drive died for good. Microsoft is not out of business, but they have said they will stop supporting Win 98SE shortly and once my older computer goes, some more of my favorite games go by by because they will not play correctly on Win XP.

It is not a conspiracy theory. If Canon goes out of business it will be way far too late to worry at that point. And your conversion of 8-track to MP3 results in a loss of quality, which may also ocurr when you upgrade your MP-3 to the next format that you are not worried about. RAW is the original set of data. Canon has converted from CRW to CR2 format already and I suspect at some point the will convert their format again. Do you really believe that a LARGE CORPORATION will continue to support an old format forever?

As to small companies producing solutions. Three factors compicate this. First is the ability of the individual to find the small companies, although the net exists and yes I can use google searches to maybe find the small company, I have to know what to look for first. Second factor is the desire of someone to produce the software and possibly hardware needed to continue these upgrades in a manner that results in no degredation of data over time. Third is DMCA and similar laws which may make it against the law to convert the file.

This isn't paranoia or conspiracy theory, this is dealing with reality and trying to correct a behavior early enough so it does not become a problem.

I deal with Advanced Sensor technology for the USAF on a regular basis and when a sensor we use no longer has common components, the price of support for that sensor starts increasing exponentially. Proprietary formats are costly and risky, Standard formats save money for the consumer and lower risk. So I think I have a pretty good handle on this issue and I would rather have a standard format for RAW images then continue having proprietary RAW formats.

Since I started working computers, I have had scanners, graphics cards, sound cards, ram, and a lot of other things become not compatable with the new version of the computer and had a lot of software that either won't work with new computer or won't work with old computer (Photoshop CS comes to mind for recent annoyance in the software department not working with my older Win98SE computer)

You don't have to agree, but I think you are a bit naive in your beliefs as they apply to the majority of people.

BTW: I prefer the WMA format to the MP3 format for music as I can compress the file smaller and get better quality out of it. I doubt I would use it as an archive format though because of the prevalance of MP3 as a standard. And I don't have an 8-track player so no point in bothering with that idea (okay I don't even have any 8-tracks anymore either, got rid of them when cassette players could play the reverse side). I have already converted all of my 20+ cassettes to CD, although I still have the cassettes.


The above is just my opinion,

[Todd Jacobsen]

I think we should be pretty happy that Canon has not gone the Ni#on route and encrypt WB data. Although all raw formats are encryption to some extent, Canon has authorized Adobe's "hack" at their formats. Ni#on has not (for the D2x nef data).

Adobe has had a long standing policy of ending free updates to "expired" products. Now that CS2 is out, CS is expired - and along with the ACR that comes with it. Since there is not an update to PSE3.0, Adobe continues to update the PSE ACR. Since the ACR file utilized in CS and PSE3.0 were "update-able" prior to CS2, I'm a little surprised that Adobe took the hard stance on PSCS ACR updates - you'd think the PSE use-able data (which includes new RAW formats - new cameras -recognition) would still work with PSCS.

[kenyc]

I think the truth is somewhere in-between. If canon goes out of business we have other things to worry about in addition to our images. (Now if Microsoft goes out of business and our programs won't run....:) ).

I think we're okay for the foreseeable future, probably for my lifetime as far as being able to access our data (which is what it is) and there are always ways to get at it and convert it if we don't delete it. I'd bet most photographers don't shoot raw anyway.

Still I've tried a few shots in raw and want to do more. The thing I think a standard would get us is better/cheaper software and hardward. That might be a reason to support it. If all cameras used the same raw format, then we could have more competition and potentially lower prices as well as more effort in general on improving the format and tools to use it.

KAC

[Curtis N]

The propagation of this "Chicken Little" attitude is motivating some naive people to continue shooting film in the fear that digital files will be worthless in a few years. Now there are some legitimate reasons to shoot film, but this isn't one of them.

I could site numerous examples of competing "proprietary" formats of emerging technology, going back to Edison vs. Westinghouse (AC vs. DC) and beyond. Eventually the superior format is determined by the marketplace, protected by patent for a certain length of time, shared through licensing agreements, and eventually becoming ubiquitous. Perhaps that's small consolation for owners of Betamax video recorders or DC motors, but it's always been one of the risks of early adoption of new technology.

Similarly, the RAW file format issue will eventually sort itself out, just as sure as this computer is plugged into an AC outlet. It will happen with or without a letter-writing campaign to camera makers.

Let's relax and make some pictures!

[CyberDyneSystems]

The moment the "RAW" file is no longer camera specific.. it will no longer be a "RAW" file anymore.

RAW is the lack of an image file format...
It becomes an image file format when it's converted to tiff, jpeg, .psd etc.

[Longwatcher]

I appologize I did not realize that so many people felt this was a non-issue. It is very important to me as I shoot only RAW as my primary image (although I also shoot M2 or small size jpeg to provide a shot to the models I shoot) and I use a 1DsMkII because I want the highest quality for my images as possible within my means to afford. And I want them to be available to me in the future forever.

I feel that if everyone let Canon know we would like them to adopt a industry standard format for RAW images with the other camera manufacturer's then this will make it easier on ME in the future to be able to access my images in the future, but since you all don't care; Please accept my appologises for bothering you with what in my opinion was an important issue.

Anybody that is still shooting film just because of this issue is being a bit silly. Anybody contemplating getting rid of a perfectly good camera or not buying one because of this issue is also a bit silly. However I feel encouraging a standard is a good thing. Again sorry you all don't feel the same way I do. I bought my Canon camera's because over all they were the best solution and if that were still the case even if they had an encrypted file system that I could not access with anything but Canon software, I would still buy the camera, but it would definately have to have something to make up for that annoyance.

Just my opinion,

[CappuccinoDavid]

Well I use PhotoShop right now for my RAW, yes I have Canon's software, but I like having one program for all. Plus that is why I use Canon and it not only because I can spell it. Before I went to Canon I used Nikon (yes I can spell it also) because it was a pro camera and I knew they will be around for a long time. The ones who will have the problem is the off brand companies, the fly by nighters. So I don't think I have to worry and plus by the time my stuff is outdated I will be upgrading by then (I hope).

[Rob612]

Standard formats save money for the consumer and lower risk. So I think I have a pretty good handle on this issue and I would rather have a standard format for RAW images then continue having proprietary RAW formats.


Wait a second. A Standard, either official (i.e. IEEE1394 or 802.1b/g) or de facto (i.e. *.doc or *.pdf, both proprietary) is a a thing. The proposal for an "open" standard is a quite different issue, IMHO.

[robertwgross]

Much ado about nothing.

Granted, I have several spindles of CDR disks full of RAW files. However, I also have 150GB or so of TIF image files. The TIF images look a little different from the TIF files from the late 1980s, but otherwise, TIF is a nice universal format.

What happens if I get a substantiated indication that TIF is going away soon? I'll do a mass conversion to whatever the standard format of the day is becoming.

No big deal.

---Bob Gross---

[tim]

How many GB of TIFFs do you have Bob?

[robertwgross]

How many GB of TIFFs do you have Bob?

As I just stated, at least 150 Gigabytes of TIF files on one hard disk and copied on another hard disk and copied somewhat on DVD disks. Then maybe 200 CDR disks of RAW files. Then more miscellaneous JPEG files for the web.

I won't even try to count the filing cabinet drawers full of 1.4MB floppy disks, ZIP disks, 5-1/4" floppy disks, and everything in between. It would not surprise me if some 8 inch floppy disks fell out as well. There is a stack of various old hard disks that probably still work and are full of image files.

---Bob Gross---

[Tom W]

The moment the "RAW" file is no longer camera specific.. it will no longer be a "RAW" file anymore.

RAW is the lack of an image file format...
It becomes an image file format when it's converted to tiff, jpeg, .psd etc.

Very interesting quote - perhaps what is really needed is a 12-bit high-quality universal JPEG format. Canon, Nikon, et. al., all put proprietary information in their RAW formats. Each does so to try to gain competitive superiority (faster processing, better color, more file information, or whatever). Canon has changed and improved their RAW format since earlier versions, and probably doesn't want to be stuck in a universal RAW format at this time.

But yes, the high-quality universal JPEG, TIFF, or whatever format would be nice for archiving files if it would guarantee forward compatibility. Kind of like the C-41 process that is used for print film. Very standardized, allows many participants in the process. I'm just not sure that I want manufacturers to stop progressing forward on the RAW format yet. Maybe when the digital imagery industry is more mature and there's little room left for improvement.

[Roach711]

Being a professional computer geek I see no *long term* (meaning ten - twenty years or so) hope of having a standard RAW format. These things we call "cameras" are actually computers and in the computer world standards aren't standard for very long. JPG and MP3 have held up fairly well but most would agree that they are not the optimum method available. If you want top quality you use another (usually less popular and shorter-lived) format.

Technology *will* progress (way too quickly for my bank account to keep up) and better ways of capturing images will be invented, so even todays "standard" format will become old technology and be replaced with new (and better) technology all too soon. Fortunately, the big software makers (usually) include converters for legacy formats so I'm not very concerned about being able to access my files down the road. I will, however, make sure that Photoshop CS3,4,5,6,7 etc. will read my old files before tossing my existing program CDs. Archiving film was certainly simpler in some ways, but given digital's advantages I'd never go back to it.

I guess my point is, if we want top quality we'll always be on the "bleeding edge" and will just have to put up with periodical re-conversion of our archived images.

Just my 2 cents - Da Roach

[Roach711]

I am very interested in Adobe's DNG format. I like the idea of taking the RAW format-of-the-day and converting it (hopefully without loosing any of RAW's post-processing advantages) to a common format. It would certainly be a step in the right direction.

Da Roach

[KennyG]

This open RAW format idea is simply rubbish, People claim Linux is open, that is also rubbish as there are so many differences between vendor's proprietary versions and software that will run on some and not others, it is about as open as as a locked prison cell door.

Why should a camera manufacturer perhaps limit themselves to performance or quality just to stick to a so called open standard? If Canon came up with a RAW format that produced better images than an open standard would you prefer to stick with something inferior because it is open? Somehow I think not.

Why not insist on all car engines being standard? Shouldn't the pistons from a Ferrari 360 fit into a Hyundai? The RAW standard debate is equally as stupid.

[Longwatcher]

I tried to let this go, but keep coming back to

Why is a standard not a good thing to want from me the photographer's and consumer's view point.

Please explain this to me in a language I can understand. Perhaps because I am a bit older then most I am having trouble with this new math concept of cross-platform standards not being a good thing for consumers.

[Drk Orange]

Just a quick point to all those worried about losing current RAW support in the future - I challenge you to come up with *one* popular image format that is not currently supported by Photoshop. If the popular image converters/processors currently read it, the feature is not going to be removed.

I can even open my *.iff images made 20 years ago on my Amiga.

[Harry Settle]

Totaly worried about the wrong stuff here. It's not the software that you have to worry about becomming obsolete, it's the hardware. That's what places like NASA have found out while trying to recover old data. The data is fine, just don't have anything to read it with. Think about this when archiving to CD, DVD and maybe even hard drives.

[slin100]

Longwatcher, like you the reactions of the respondents so far has me nonplussed. I'm not sure why people are not more worried. I'm not sure if the following is true, but I heard that it's hard to find decoders for Canon D2000 RAW files. The D2000 doesn't seem to be supported by ACR. If that doesn't give one cause for concern, it should. Do people really think Adobe will maintain support for, say, the 10D in perpetuity?

People may not realize this but the WB data for Canon RAW files cannot be decoded by ACR. Ever notice that the As Shot color temperature in ACR, while very close, never really matches the Canon RAW converters?

People who say that they'll simply convert their files when it becomes necessary must be really on the ball. I think we are more likely to be in the same position as Bob Gross, with cabinets full of old file formats stored on old media, long after either the format or media can be decoded or accessed.

Those that cite the TIFF file format as a counterargument that this is a nonissue should realize that TIFF is a standard format, unlike RAW. I've also read that one disadvantage with TIFF is that the stored image data is usually gamma-encoded, just like JPG. Gamma-encoded data is less amenable to post processing than linear-encoded data.

[slin100]

Just a quick point to all those worried about losing current RAW support in the future - I challenge you to come up with *one* popular image format that is not currently supported by Photoshop. If the popular image converters/processors currently read it, the feature is not going to be removed.

See the previous post. Canon D2000 RAW files. Look here (http://www.adobe.com/products/photoshop/cameraraw.html) for a list of supported RAW formats.

[tim]

See the previous post. Canon D2000 RAW files. Look here (http://www.adobe.com/products/photoshop/cameraraw.html) for a list of supported RAW formats.

That's not a popular image format, it's a proprietary raw format from what I presume is an old camera.

[slin100]

That's not a popular image format, it's a proprietary raw format from what I presume is an old camera.
All Canon RAW files are proprietary. And, yes, the D2000 is an old digital camera, but not really that old in the grand scheme of things, such as one's lifetime. It came out in 1998. 20 years from now, do you think your 20D RAW files will still be popular?

[tim]

All Canon RAW files are proprietary. And, yes, the D2000 is an old digital camera, but not really that old in the grand scheme of things, such as one's lifetime. It came out in 1998. 20 years from now, do you think your 20D RAW files will still be popular?

Don't know about popular, but I don't think it'll be read by much software in 20 years.

[slin100]

Don't know about popular, but I don't think it'll be read by much software in 20 years.
Which is precisely the point about why you should be concerned.

[tim]

I don't think standardisation of RAW formats is a good idea. If it's standardised it's not RAW, by definition. Perhaps a better idea would be to offer DNG as well as their own RAW format on each camera?

[lostdoggy]

d2000 is that a EOS-1n Body with a Kodak Back??? Kodak DCS520 is still supported but not D2000!!!

[lostdoggy]

That camera was beyond most comsumer as well as most pro photographers at the time. It was mostly used by Commissioned Sports and News Photographer, but I think they have move on.

[CyberDyneSystems]

I am very interested in Adobe's DNG format. I like the idea of taking the RAW format-of-the-day and converting it (hopefully without loosing any of RAW's post-processing advantages) to a common format. It would certainly be a step in the right direction.

Da Roach

I would LOVE it if we could be guaranteed that no RAW functionality and image quality would be lost... but I don't see that guarantee being made.

ACR is allready hopelessly lost.. (at least the ACR in CS,. I don't have CS2 yet) it ignores half the exif data and camera settings in the RAW files it imports, applies settings that are not in the camera,. and can't even read when we have the Adobe color space embedded on it's own.

If this is Adobe's best attempt... how can we ever expect it's DNG to have all the right info in it after the conversion?

[lancea]

Totaly worried about the wrong stuff here. It's not the software that you have to worry about becomming obsolete, it's the hardware.
YES! I started reading through the thread and was surprised just how long it took to find someone who identified the real problem. I posted similar a few weeks ago. I have an 8" floppy, tens of 5 1/4" and hundreds of 3 1/2" floppies. Most of them are unreadable and I haven't had anything that would read the big disks for a long time. I have CD-RW discs and CD-R discs that are unreadable (thankfully not many, but it does happen).

CDs have insufficient capacity to hold a realistic number of RAW or TIFF files, and single layer DVDs aren't all that much bigger. Dual-layer DVDs are expensive and the writers aren't so reliable. It's almost certain that as sensors get "bigger" there will be more and more shots that never leave a hard disc - and hard discs don't last forever. I remember the days when I used to back everything up to floppy, but when it took 50 floppies I stopped backing-up all but essential files. Backup has always and will always be a significant problem.

In short, it's very likely you will be able to open your files in 20 years - provided you can still access the file. As for 50 or 100 years, perhaps we don't care too much anyway, but it would be nice to think our shots are at least as long-lived as our great grandad's :)

[Poco]

I have to think you are very naive in your assumptions.

I am happy you still have a functioning 8-track player and a 5.25inch floppy drive available to you. I can still do cassette and vinyl myself, I was given all of my Grandmother's records because I was the only one in the family with a working turntable left. A lot of these records are classics that have no significant market value to be produced on CD, so I need to do it myself if I want to be able to play this music in the future.

However, I can't run several of my favorite DOS based games on either of my computers anymore because of clock-timing problems. I can't without paying someone a LOT of money get someone to convert a laser disk to DVD or CD format. My old 250MB zip drives are pretty much useless unless I can snag someone with an old drive that still works, luckily I convert data to CD before my drive died for good. Microsoft is not out of business, but they have said they will stop supporting Win 98SE shortly and once my older computer goes, some more of my favorite games go by by because they will not play correctly on Win XP.

It is not a conspiracy theory. If Canon goes out of business it will be way far too late to worry at that point. And your conversion of 8-track to MP3 results in a loss of quality, which may also ocurr when you upgrade your MP-3 to the next format that you are not worried about. RAW is the original set of data. Canon has converted from CRW to CR2 format already and I suspect at some point the will convert their format again. Do you really believe that a LARGE CORPORATION will continue to support an old format forever?

As to small companies producing solutions. Three factors compicate this. First is the ability of the individual to find the small companies, although the net exists and yes I can use google searches to maybe find the small company, I have to know what to look for first. Second factor is the desire of someone to produce the software and possibly hardware needed to continue these upgrades in a manner that results in no degredation of data over time. Third is DMCA and similar laws which may make it against the law to convert the file.

This isn't paranoia or conspiracy theory, this is dealing with reality and trying to correct a behavior early enough so it does not become a problem.

I deal with Advanced Sensor technology for the USAF on a regular basis and when a sensor we use no longer has common components, the price of support for that sensor starts increasing exponentially. Proprietary formats are costly and risky, Standard formats save money for the consumer and lower risk. So I think I have a pretty good handle on this issue and I would rather have a standard format for RAW images then continue having proprietary RAW formats.

Since I started working computers, I have had scanners, graphics cards, sound cards, ram, and a lot of other things become not compatable with the new version of the computer and had a lot of software that either won't work with new computer or won't work with old computer (Photoshop CS comes to mind for recent annoyance in the software department not working with my older Win98SE computer)

You don't have to agree, but I think you are a bit naive in your beliefs as they apply to the majority of people.

BTW: I prefer the WMA format to the MP3 format for music as I can compress the file smaller and get better quality out of it. I doubt I would use it as an archive format though because of the prevalance of MP3 as a standard. And I don't have an 8-track player so no point in bothering with that idea (okay I don't even have any 8-tracks anymore either, got rid of them when cassette players could play the reverse side). I have already converted all of my 20+ cassettes to CD, although I still have the cassettes.


The above is just my opinion,

I agree with most of your opinions and observations, but I think we have to be careful to compare apples to apples. I, and others, don't agree with your conclusions. I think the issue here is that your examples don't apply well to file formats.

Yes, hardware formats (LP, 8 Track, LD, Zip Disk) die and are difficult to continue to use. Yes, software dies and will eventually not be runnable, though if someone really wanted to make it run, they can using some form of emulation (I can play C64 games on my PC if I wanted to).

First, RAW is not software, it is a file format. So comparing it to the life cycle of software doesn't make any sense. Second, RAW is not hardware. The data on your LP/LD/Zip Disk/8 track is not dead, just the storage format. A better comparison would be to say that the old Wordperfect files on those Zip Disks cannot be read, but, of course, they can if you want it badly enough.

No doubt someday DVDs will be hard to read - but the files I'm putting on them do not have to be. There is always the possibility that Canon will stop supporting existing RAW formats, or Nikon, or whatever. That doesn't mean that those files cannot at that time or afterwards, continue to be used or converted into something else.

With something as popular as Canon RAW files you have to believe that someone out there will want to convert them to format X as much as you do. There are a lot of programmers who have a lot of RAW files (I know many) who are perfectly capable of doing it themselves. Do not worry about that.

The only thing you should concern yourself with is making sure that your old files are safe and accessible. You should make multiple backup copies and, more importantly, keep them on relevant hardware. When you get your BlueRay writer (or whatever) in a few years make sure to copy all your old backup CDs and DVDs.

As long as you make sure that you can get the files, someone will be able to get the information out of them.

[sdommin]

Why is a standard not a good thing to want from me the photographer's and consumer's view point.

That is a good question, Longwatcher, and I would like an answer from our "anti-standard" experts, too. Exactly how much better is a photograph from Canon RAW than a photograph from Nikon RAW or Olympus RAW, etc.? How would we all suffer if somehow there only existed one RAW standard?

[Jesper]

I didn't read the whole discussion in the posts above, but I agree that we need an open RAW format!

Nikon is TOTALLY on the wrong way with this, with their encrypting of white balance data in the RAW files of certain cameras. I'm not sure, but as far as I know, you don't even get fully functional RAW conversion software with your camera from Nikon, you have to buy it separately (Nikon Capture). I'm glad that we Canon users at least get things like DPP for free from Canon.

[Harry Settle]

Why is a standard not a good thing to want from me the photographer's and consumer's view point.


I'll go out on a limb here and suggest that companies aren't worried as much about the consumer as they are at protecting their pocketbook. MONEY!!!

[stampchick724]

There has always been a way to convert one format to another. with exception of 8 track tapes to cassette, I cant think of a single thing I cant, one way or another.


8mm to VHS, VHS to DVD, DVD's to the future latest and greatest (more info on a DVD sized format).

slides to negatives, slideprojectors to digital (power point) presentations, negatives to digital files.

film processors in cans, film roll processors, custom printers in darkrooms, retouchers, spot corrections, C-41, R-2, copy negs, dodging burning. All to PHOTOSHOP.

But, then I DO agree with this....a universal format will dominate. Remember BETA tapes??? but only for a moment in time. just wait, in the blink of an eye things will change!


in december I has a cassette tape player and a camera from 1979. Now, I have SIRUS radio, CD player, an ipod (also to play in vehicle), a digital camera.

[slin100]

There has always been a way to convert one format to another. with exception of 8 track tapes to cassette, I cant think of a single thing I cant, one way or another.


8mm to VHS, VHS to DVD, DVD's to the future latest and greatest (more info on a DVD sized format).

And why, do you think, you have been able to do this? Because all those formats are standardized. RAW is not. I amazed that people want to rely on grassroots efforts to decode proprietary formats. Do people really want to entrust their personal data in that manner?

[Belmondo]

All this has me thinking that that maybe I should be saving my important shots in multiple formats. RAW, TIFF, and high-quality JPEG.

Gads, what a nightmare.

[Longwatcher]

Some comments.
I have already developed a solution to the hardware format issue. I copy my files to the next format as needed. While a nuisance in terms of time and effort, the result does not cause any lose of data once it reaches a digital versus analog state. Thus I can take a jpeg file from 3.5" floppy to CD to DVD to super flash whatever and the data itself does not get corrupted by the process of converting hardware formats, since the data is in the digital realm already.

Changing file formats on the other hand such as CR2 to DNG may entail a loss of data just by the conversion process. One of the reason jpeg is not a good archive format is that it is inherently lossy by design. Tif on the other hand is a relatively good archival format in that the data tends to be kept intact (although you start losing metadata) in the conversion to TIF. TIF is just a bit large. Since the raw file is the orginal version of the data it has not incurred any loss yet from file conversion, I would prefer not having to convert file formats if this can be avoided. If it can't be avoided then I want it to be a lossless transfer. Currently this is not possible, because the camera companies don't want it to be. We the consumer can get them to change their mind. I suppose I could live without a raw standard if I knew I would be able to losslessly convert the data to a new format years down the road, but I am of the opinion a raw standard would be a better mechanism.

A good standard RAW format would keep both the original image data and the metadata intact in a manner that it could be used between camera systems. It should also handle the future which is multi-band image systems (somewhat like foveon sensor)

Lastly.
A RAW image format is one that records the results of the sensor data in a manner that can be utlized by other systems. Before that it is generally considered RAW sensor data and is not in itself useable except by scientific instrumentation. So a Standard RAW image format is possible. A standard RAW sensor format between camera systems is not.

At my dayjob I work with a file format called NITF (see if you can find support for that on PS CS2) it is an international standard used by NATO (can't remember the STANAG #).
NITF is mostly similiar to TIF, however NITF can be used for almost any RAW image data that the US uses because it has a lot of flexability built into it. It is currently on version 2.1 as far as common use is concerned (it may be on 2.3 or 3.0 in the labs). The USAF mandates that all Air Force imaging systems must use the NITF standard if they wish to trasnmit images to the USAF's ground station. This was mandated so we did not have to use 20+ different proprietary file formats in our ground station. This makes it a whole lot cheaper on the taxpayer and on the analysts. The other reason is that getting the images from sensor to the ground is only a small portion of the process of getting information to the front lines.
Now as to limitations of standards, NITF is an excellent example. Today we use it for EO, IR, SAR and MSI images. However we are currently having some problems with HSI data as the development folks found they needed some metadata that NITF currently doesn't support in order to properly process HSI data sets. The solution is one of two. First they are trying to find out if NITF can be modified to accept the metadata they need. If we find that it can not for some reason, then we will have to create a new standard to handle this form of data, but we will have a standard again that will accomodate all of the above image types coming from any sensor in our inventory.

This is probably why I would like to see the same thing in the commercial realm as we found it to be a necessity in the long run to save money and problems.

I can see better where some of you are coming from now, but somebody still needs to explain why having a standard file format is not a good thing.

For those that don't know the acronyms
EO= Electro-optical (normally means visible spectrum images, but technically means any digital imaging sensor using optics)
IR = Infra red
SAR = Synthetic Aperture Radar (cool stuff - see through clouds)
MSI = multi-spectral imagery (EO images taken in more then one band of the spectrum - thing color versus BW)
HSI -= Hyperspectral Information (EO images taken in 100 or more bands of the spectrum - very cool stuff)

To me this is an important issue obviously, but if I can be satisfied in my primary needs to be able to access my data without loss years/decades from now, I will be happy. As mentioned I have an okay solution to the hardware problem. It is the software portion I am worried about.

[Steven M. Anthony]

I don't understand the issue here. I shoot raw. I have PS CS, and the raw images open up just fine in it. If Canon and/or Adobe go out of business tomorrow, my 10D will still shoot raw files and PS CS will still be able to open them.

[tim]

I don't understand the issue here. I shoot raw. I have PS CS, and the raw images open up just fine in it. If Canon and/or Adobe go out of business tomorrow, my 10D will still shoot raw files and PS CS will still be able to open them.

What about in 10 or 20 years, will Photoshop version 15 support 20D RAW files? I doubt it.

[Steven M. Anthony]

Maybe not--but the copy of PS CS on my computer will, just like the version of MS/Word I have on my HP Pavillion (166MHz Pentium [1]) still works like the day I bought it.

[tim]

Maybe not--but the copy of PS CS on my computer will, just like the version of MS/Word I have on my HP Pavillion (166MHz Pentium [1]) still works like the day I bought it.

That computer will be an antique by then, the machine you have then might not run windows or windows apps. Of course it might, i'm just pointing out the problem with RAW as an archival format.

[Poco]

I wasn't saying that people shouldn't convert their files into something better. I was saying that it isn't an emergency as some are suggesting. If there is a common format in a couple of years that can contain all your RAW data without error or loss, then go ahead and convert your files. I might. But I'm not freaking out about it. Perhaps when I move my backups to Blue Ray or HD DVD I will make a DNG (or whatever has replaced it by then) copy of all the files.

Also, let's not forget, just because a file format is a standard (or at least well supported) does not mean that all the data in the file can be processed by any application. Even some of the Exif data in our current RAW and JPEG and TIFF files is lost or inaccessible in some applications. In fact, the more "common" a file format is, the more likely that it has defined how to put arbitrary chunks of information inside it that other programs may not recognize. I haven't read the full DNG spec but I suspect it has something like this (particularly if it contains RAW Exif data). While this is good for extensibility, it means that you could just as easily have data in your DNG file that Photoshop does not recognize and loses when you convert it, especially if that DNG came right our of your camera. So even a single file format does not solve all the problems unless that file format has standardized every way that someone might use it.

Oh, and don't lump me in with those who think that they will be running PS CS in 10 years. Though I wouldn't go so far as to say that PS 15 will not support existing RAW files. File formats are very easy to support for an application like Photoshop and it would be to their benefit to continue to support all the formats they support now and more.

[S230]

This just came up and thought it may be some relivance to the topic. Now that such a big organization is recognizing the RAW format, it's probably going to stay for a while.

Microsoft to add RAW support
Wednesday, 1 June 2005 13:05 GMT

http://www.dpreview.com/news/0506/05060102microsoftraw.asp


http://photography-on-the.net/forum/showthread.php?p=579034&posted=1#post579034

[Jon]

I have to agree with CDS (not difficult, actually). A "standard" RAW format will get extremely awkward for cameras with non-standard sensor types (Foveon, Sony's R-G-B-Emerald for instance), and as the technology advances we'll start seeing more and more of these, which will be difficult, at best, to graft onto DNG. If you want a comprehensive, lossless format, consider JPEG2000, which has only a name in common with the JPEG we all know and love(?). I don't see TIFF going away any time soon either.

[slin100]

Take a look at the DNG spec. It doesn't preclude new sensor types. It can support sensors with more than 3 color channels. Even with 3 color channels it doesn't assume that they're RGB.

The DNG spec is not that restrictive. It makes allowances for storing proprietary data. In some ways, it's a glorified TIFF format.

[PacAce]

About 50 years ago, my father took a lot of pictures of me and my sibling when we little. A few years ago, I went through my father's negatives and slides, looked through them and digitized a few of them to send to my brother and sister to share.

Today, I'm shooting picture of my grandson in RAW format with my 10D and 1DmkII and all the RAW files are archived to CDs and DVDs. Fifty years from now, when my grandson finds these CDs and DVDs locked away someplace, what is he going to find? Will he be able to experience the same feeling I did when I saw images of myself as a baby on a negative? Will he even be able to access the images on the CDs (assuming the data burned there lasts that long)? I've often wondered about this.

[Chazs]

Standards change: the standard highschool math class 20 years ago was a far cry from what it is today (for better or worse). Also, when the random array sensor is in production (to eliminate banding, moire patterns, high noise patterning, etc) the data won't be linear and so an entirely new type of raw data information coding will be necessary. Out with the old RAW and in with the new.

[slin100]

Standards change: the standard highschool math class 20 years ago was a far cry from what it is today (for better or worse). Also, when the random array sensor is in production (to eliminate banding, moire patterns, high noise patterning, etc) the data won't be linear and so an entirely new type of raw data information coding will be necessary. Out with the old RAW and in with the new.
Again, a well designed file format doesn't preclude advances in sensor technology. As far as I can tell, DNG is a step in the right direction. It may not be perfect but it's open-ended enough to accomodate growth. Think of DNG as a file cabinet with a few labeled folders. One of the folders is labeled "Image Data". The image data is a piece of paper that goes in this folder. There are no restrictions about what is written on the paper. You can even use more than one piece of paper (i.e. a new technology) if you want.

Can anyone point out any flaws in this analogy?

[Poco]

Again, a well designed file format doesn't preclude advances in sensor technology. As far as I can tell, DNG is a step in the right direction. It may not be perfect but it's open-ended enough to accomodate growth. Think of DNG as a file cabinet with a few labeled folders. One of the folders is labeled "Image Data". The image data is a piece of paper that goes in this folder. There are no restrictions about what is written on the paper. You can even use more than one piece of paper (i.e. a new technology) if you want.

Can anyone point out any flaws in this analogy?

There is no flaw with that analogy, but it is no different than the analogy you could use when refering to the RAW format. The only difference is that the file cabinet is of a different design but they hold the same paper.

The problem with that logic is that, just because a file format "can" support the data does not make it any more likely for applications to support that format of data in that file in the future. In fact, the open endedness of it is what makes it harder for others to fully support it, because full support is a moving target.

What I'm proposing is that you will have just as hard a time loading a DNG file you create today with your RAW Canon data in it than you will have loading a RAW Canon file (CR2). I'm not saying it will be a problem, but they will both be similar. Actually, now that I think about it, it may be harder to read the "special" DNG file than the CR2 file because the CR2 will be more restricted in the variations of data within it.

Take BMP as an example. Did you know that the way the file format is defined it can handle data of any bit depth per channel. The bits-per-channel is defined in the header so in theory you could have 9 bits per channel. How many applications do you think would be able to read a BMP file I created that defined 9 bits per channel? None.

So it is not just about what the format can support, it is about how the file format is used. If you store obscure data in a DNG file, no one will be able to interpret it.

As I've said before in this thread - Just make sure that you can get access to the files on whatever media you have it stored. It is more likely that the CDR with your pictures will be destroyed or unreadable in 50 years than the data in the file will be inaccessible. Fortunately, the size of the backup media keeps growing. As soon as I got my DVDR I could put all of my CDR backups on to one or two DVDs. I expect that the next recordable format I get will take all my DVDRs and so on. In 50 years I expect to have a keychain that will hold every picture I have ever taken - ever.