What TIFF compression?

[dsze]

Which TIFF compression is the best to use? What are most people using. I've been using LZW, but not for any particular reason. OR, are people leaving their TIFFS uncomped?

-daniel

[Scottes]

I have found that for most images it's best to leave them uncompressed. I don't like ZIP compression because of it's lack of support. LZW is so simple that it often make DSLR images larger than the original. The randomness of noise is enough to make LZW useless.

If I have an image that I've done noise reduction, and it's got a large expanse of sky or water or such, then I might use LZW.

If I could use ZIP I probably would use it all the time - ZIP compression is much smarter than LZW, so it would have a better chance of actually compressing. But then I'd be worried about the compression / decompression time required. I'd rather have big files than wait 5 seconds for a file to open or save.

So mostly now I just leave it off, and save straight TIFF.

[Belmondo]

Ditto.
I use no compression in TIFF at all.

[drisley]

I use LZW as it really works well to keep the file sizes much smaller.
However, if you use 16bit tiffs, for some reason LZW does make the files sizes bigger! That is odd isnt it.

[maderito]

The last time I checked, the LZW compression algorithm copyright protection was still held by Unisys and thus subject to patent protections - e.g. fees have to be paid to use the alogirithm in software programs.

LZW compression is optionally used in TIFF files and always (I believe) in GIF image files. This has caused major grief for many - especially software developers.

The bottom line is that some applications cannot read TIFF files that are LZW compressed. This problem typically arises in shareware/freeware products for obvious reasons.

I actually was impressed with the file size reduction I sometimes found with LZW compression until one of my software programs (IMatch) coughed and couldn't display the files. :evil: I've since stopped compressing TIFF's.

The patent is supposed to expire - but I don't know the latest.

[CyberDyneSystems]

I've never try to compress a Tiff..

When I get mad I like to let the whole world see... :wink:

[Cadwell]

I've never try to compress a Tiff..

When I get mad I like to let the whole world see... :wink:

:lol: I don't suppose you guys from the Western side of the pond know this... but there is a fairly well known British racing driver / TV presenter named "Tiff Needell". I kid you not. He's rubbish (as a racing driver), but he does exist.

[maderito]

I use LZW as it really works well to keep the file sizes much smaller.
However, if you use 16bit tiffs, for some reason LZW does make the files sizes bigger! That is odd isnt it.
Yes, odd. This implies that 16-bit images contain a lot of noise. :?

Maybe there are random 0,1's in the high bits of a 16-bit file (12 bits of real image data). Maybe it happens after in-camera or post processing image data manipulation (like gamma correction, levels adjustment, etc.).

Wait a minute - didn't we just talk about noise levels on another thread. :)

[Scottes]

I use LZW as it really works well to keep the file sizes much smaller.
However, if you use 16bit tiffs, for some reason LZW does make the files sizes bigger! That is odd isnt it.
Yes, odd. This implies that 16-bit images contain a lot of noise.

Actually it just means that 16-bit files have more variations.

LZW is stupid. An 8-bit RGB pixel 251/128/191 will only match another pixel at 251/128/191 (in a row actually). Well, if you have a 16-bit file then the first pixel is really RGB 64256/32768/48896, and this will not match a pixel that's RGB 64256/32768/48897. Now these two pixels will look identical to the human eye, and they *are* identical in 8-bit mode, but to LZW the are not the exact same, thus they are completely different.

So LZW will now make two table entries of 1 pixel each. Since a table entry takes several pixels this makes the "compressed" file even larger.


BTW, JPG would just take these two pixels and actually make them the same color, thus allowing JPG to compress them. And JPG is a lot more evil than this. 251/128/191 could match 253/126/189 depending on compression level. Ugh.

[dsze]

Ok, so maybe I'll stop compressing my Tiff's. :)

[drisley]

I would keep compressing them.
The compression is lossless, so you lose no quality.
But, unless you save 16bit Tiffs (which I dont after doing all my colour correction etc), you can decrease the space used by well over 50% (often closer to 66%).
If I run into a program that doesnt use LZW compression (which is rare) I just open in photoshop, and save without compression, and then use the program.
This sure beats spending more $$ on CD's and DVD's.

[drisley]

I just did a test on a full sized 300D image using 8bit and 16bit modes, uncompressed, lzw, and zip. Here is what I got.

8 Bit:
Uncompressed: 18.5MB
LZW: 6.6MB
ZIP: 6.7MB

16 Bit:
Uncompressed: 37MB
LZW: 46MB
ZIP: 34MB

So, if you are using 16bit Tiffs, compression really wont help, and in the case of LZW, you will end up with BIGGER files sizes! :shock:
Since I always convert to 8bit for files srorage, LZW really does work well.

[dsze]

The randomness of noise is enough to make LZW useless.

Scottes, could you expand on this. Does compressing with the LZW alg. actually cause random noise?

-daniel

[Scottes]

And I tried one - a full-frame squirrel, so lotsa of detail, no noise-reduction on this one yet.

8-bit
None: 11.5 MB
LZW: 5.7 MB
ZIP: 5.6 MB

16-bit
None: 21.9 MB
LZW: 29.4
Zip: 22.0

It all depends on the image... I've had some go from 36 MB to over 48 MB. I got tired of guessing, so I no longer compress. Saving 15-cent CDs is not worth my time.

[drisley]

Since I shoot in 8bit mode, I would use 3x as many cd's if I didnt use compression. It's more than just the money. It's the pain in the arse trying to store all these cd's, and later trying to find the cd with the image I want. It's the difference between having to buy/store/sort 100cd's vs 300cd's. Not to mention is more environmentally friendly!
:)

[Scottes]

Scottes, could you expand on this. Does compressing with the LZW alg. actually cause random noise?

Oh, no. It's just that LZW sees the most minute differences - like random noise - as completely different. Randomness hurts compression, especially one that does Run Length Encoding like LZW.

Compressions schemes depend on patterns - it's how they compress. Something like 1212121212121212 gets encoded as 12,8 - or "the pattern 12 repeated 8 times in a row." So it gets compressed to 3 digits rather than 16.

Randomness means no patterns:
12399123891239812399 will get LZW encoded as
12399123891239812399,1 - it takes an extra byte to store this "pattern" repeated 1 time.

Now JPG would take the above info and decide that they're all close enough to each other so it won't make a difference to the human eye. So it removes information, deciding that 1, 2, and 3 are very close, so make them all 2s. And all the 89, 98, 99 stuff is similar, so make them all 99:
12399123891239812399 becomes:
22299222992229922299 and this gets compressed as
22299,4
And those why JPG is considered "lossy" because information is lost.

16-bit mode has far more chance of being random. 8-bit mode means 256 choices for each color. 16-bit mode means there are 65,536 choices for each color. So 16-bit mode is far less likely to have patterns, so compression may make it bigger.

You wouldn't see much difference if you took a compressed 8-bit file and converted it to 16-bits and then immediately compressed it. You didn't change any info, so the patterns are all the same - they're just a little longer.

Now take that 8-bit file and convert it to 16-bit and manipulate it - contrast & color adjustments, sharpening, etc. You've just changed colors all over the place, and since you did it in 16-bit mode there will be slight variations between colors now. These variations are randomness to LZW, and if you compress this file it may very well get bigger than a non-compressed version.

[dsze]

Got it. Thanks. So, back to ZIP compression then: is the main problem here the lack of other programs that will open it? I gather that ZIP is a more complex and more efficient compression Alg., so, if I'm always using PS to open images and converting to JPG or uncompressed TIFF to use elsewhere, are there any downfalls to compressing with ZIP?

-daniel

[Scottes]

If you only use PS then you're OK. ZIP is definitely far more intelligent about compression - there will rarely be a time when an image will get larger with ZIP.

However - and I'm guessing a bit here - you probably won't see much difference between LZW and ZIP on low-ISO and/or low-detail shots in 8-bit mode. And you may find a time increase, however slight, to ZIP and unZIP.

Just don't use ZIP on ISO-800 shots. :)

[dsze]

I just tried a full frame of a cows face (lots of detail including flies and wounds on the cow) :) Here's what I got. The ZIP took much longer to save than the other two:

NONE: 36
LZW: 46.7
ZIP: 35.6

-daniel

[dsze]

that was a 16bit file above...sorry..

-daniel

[Scottes]

Just don't use ZIP on ISO-800 shots.

I take that back. I just saved an under-exposed sunrise taken at ISO-3200 - can you say "noisy" ?

The ZIP was still much smaller than the uncompressed version. 33MB vs 38MB.

[Jesper]

I use LZW as it really works well to keep the file sizes much smaller.
However, if you use 16bit tiffs, for some reason LZW does make the files sizes bigger! That is odd isnt it.
Yes, odd. This implies that 16-bit images contain a lot of noise. :?

I remember reading somewhere that Photoshop has a bug that causes 16-bit TIFF files to become larger when you use compression, so maybe it's not because of noise at all, but just a Photoshop bug. :roll:

[theoldmoose]

I don't compress my TIFFs (or PSDs, if you can even do that), and I gave up on archiving on CDs a while back. They are simply too small, except for archiving the original raw images. At 6MB each, I can get about two rolls of film on a CD. For edited images either in TIFF or PSD with layers, use DVDs, instead. And go for some quality media like Verbatim DataLife Plus, that at least have some kind of archival rating.

Life's too short to fool around with media that can only save a dozen or so images, at best.

Also, if you are using 15 cent CDs to archive your images, then indeed your time is not very valuable to you (or your images). Someday you may want to actually access them again.

[Scottes]

Also, if you are using 15 cent CDs to archive your images, then indeed your time is not very valuable to you (or your images). Someday you may want to actually access them again.

Is there a problem with Memorex CDs? I get them from the company that supplies my company. Since we buy thousands of CDs at a time I get them cheap.

Don't assume too much.

[dsze]

Yes, I do back up on DVD, but I have to say that I was fairly disappointed. 4.5GB on a DVD really is not that much space. MUCH better than a CD certainly, but for the amount of time that it takes to burn 4.5 GB....and for the cost of blank DVD media, its almost more efficient to keep buying hard drives. I've got 300GB of HD space in my computer which cost me about $230.

-daniel

[theoldmoose]

There was a white paper at cdsleeves.com that was a real eye opener. Their business is in producing mailers, and other storage containers for recordable media. They got interested in media life issues, including how tough the various media coatings etc, are after getting boat loads of complaints from customers about unreadable media after distribution.

Turns out that most CDRs can be erased by just exposing them to sunlight for a few minutes. Leave the out on a desktop under a strong flourescent light source, and the data starts to disappears sometimes in a matter of hours.

The only recordable CD media that the white paper could recommend as archival quality as either Kodak Ultima Gold (now discontinued) or Mitsumi Gold (available under a different name, but same formulation).

Everything else was, to put it simply, just short-term crap. The best you could hope for, under ideal storage conditions might be at best ten years life.

Cheap CDs are OK for copies of CDs to knock about in your car, or to move data from one location to another (as long as you have other backups!), but for archival purposes, they leave a lot to be desired.

As for using hard drives, a number of folks have taken this tack, and it certainly has merit, as long as you remember to have some proper redundancy built into your storage system. It does no good to lose everything due to a drive failure, as seldom as that seems to occur these days, either.

Also, DVD write times may seem glacial, but with the 4x and now the newer 8x writers, it seems to be proportionate to the sheer amount of data being recorded. Being able to knock out a 4.7 GB DVD in 7 1/2 minutes on a 8x drive doesn't strike me as too shabby. How fast can you write the equivalent of 7+ CDs? To beat the DVD time, you would have to write entire 650 MB CDs in about a minute each.

[dsze]

...but what about the archival qualities of DVD's. What would make us think that to be any better than CDR's?

[mat]

its best not to use any compression at all, eg if you open pic in jpg the more you resave it it does lose quality, but if you save tiff files no compression you should save 100% quality each time

[dsze]

Yes, we know that jpg is a lossy format. But, TIFF is not. YOu can compress TIFF (in theory) without loss. TIFF is a lossless format. However, there are still other implications of saving w/various TIFF compressions, which is what we were discussing.