Lens Testing

[oops]

I recently purchased a Sigma 17-35 EX and need to do a better sharpness test than my initial "this is too soft, something must be wrong" reaction.

Any ideas of a good step-by-step method to test a new lens? There are probably several ways but what's your favorite?

When done, what should I expect or accept in terms of sharpness at, say, 17mm f2.8 versus 35mm f8.0? Is a huge difference acceptable?

So many people both love and hate this lens on the D30 that I want to be sure I really have a bad lens, and I look forward to learning something as well.

[Darryl]

I went through the same reaction. first the results of my test.....

http://www.txis.com/~darryl/photos/TEST/LensTest/lenstest.htm

Basically all I did was set up my lights, subject, and camera settings. Took a picture with one lens, then switched lenses and took the other picture. Since both cameras were variable focal lengths I was slightly off on getting that part exact. Can't remember now if focal length is included in the INFO display or not, but I know it is included in the EXIF information.

Hope this helps.

[Far Side]

A quick test to check a lens is to photograph a full newspaper page of small classified ads. This makes it easy to see if images are sharp to the edge of the frame.

[oops]

I hadn't considered a side-by-side but that is, after all, a conclusive test unless the test is flawed in favor of one lens over the other. I own the 28-135 IS and felt this would be as good a comparison as any.

So, today I mounted the D30 on a tripod and set AV 5.6 @35mm, ISO 100 with the Sigma 17-35 EX, RAW mode. Subject distance was about 40 feet. More EXIF appears with the images on Pbase if more is important. First, the Sigma 17-35 EX:
http://www.pbase.com/image/3490567.jpg
http://www.pbase.com/image/3490567

Now, the 28-135 IS at the same settings with IS turned off:
http://www.pbase.com/image/3490607.jpg
http://www.pbase.com/image/3490607

Both images were shot RAW with Linear 16 bit conversion (Breeze) and Linear 3.42 normal sharpening applied in PS 7. I tried to crop both as close as possible and converted to quality level 12.

Am I on the right track with my test or should I back up and get more scientific?

Am I seeing a problem with my new lens that doesn't exist (being too picky) or should I explore a remedy from Sigma and/or the vendor?

After today I can only absolutely say:

1. Your Canon lens beats the socks off the Tamron.

2. My 28-135 beats the socks off my Sigma 17-35.

3. Your subject is an absolute Babe and beats the HELL out of my Interstate battery.

Beyond this, I must check to those with more experience and await their verdict.

FAR SIDE: Great suggestion; I'll try this next for the edge-to-edge test unless everyone agrees "if the center doesn't cut it, the edges won't either".:)

[Rudi]

Oops wrote:
FAR SIDE: Great suggestion; I'll try this next for the edge-to-edge test unless everyone agrees "if the center doesn't cut it, the edges won't either".:)


Chris,

If the centre is soft, who cares if the edges are sharp? (I'm talking about LENSES here... :D)

From what I've seen so far, that Sigma of yours really, really sucks (sorry)! No other word for it. I cannot see how a chip upgrade by Sigma will improve the optics...

[Darryl]

Well both are tests could probably be more "scientific", but I think the end results would be the same...

The Canon lens is better, period.

-darryl

[oops]

Thank you all for responding and helping out! I have fired off a very nice e-mail to the camera shop and asked for a return authorization. I am still impressed by the praise so many people give this lens and would love to get a good one.

I'll ask them to test it first.:)

In parting, here's the "Blob" shot just for grins since I included this link for the camera shop:

http://www.pbase.com/image/3507814.jpg
http://www.pbase.com/image/3507814

Chris

[oops]

The camera shop called me @ work and said I could return the Sigma for a full refund, apply it toward the 16-35L, or try another Sigma. My choice.

Rudi, I noticed you have a Tokina 19-35 with which you are very happy. If it is the AF193C listed on B&H for $189 with a $20 rebate this would be quite a deal.

Can I talk you into a simple lens test or a close crop example?

The 16-35 is tempting but @ $1600 you wouldn't want to make a mistake.

[Rudi]

Chris,

Yes, the Tokina is really cheap! It is a slower lens, though (f/3.5-4.5), compared to the Canon 16-35 L. I am SURE that the L lens will be the better lens, but much pricier! :)

I will shoot a comparison between the 28-135 IS and the Tokina. Give me about 24 hours to post the photos. (I'm off to work now).

I lucked out with the Tokina. I knew a guy who had one, and I knew his photos were sharp, so I bought one for myself. The lens has a good reputation on the net, too, although I did not find this out until AFTER I bought mine...

Good news about your refund! Now all you have to decide is: Do you want that extra 3mm at the wide end (a BIG difference at that end!) for only $1400 extra? :D

[Rudi]

Chris (and all),

here are my (not so scientific) results, comparing the 19-35mm Tokina cheapie to the Canon 28-135 IS lens. Please keep in mind that I was in a hurry and both shots were handheld. The following are crops from the full frame, at 100% magnification. You are seeing the actual pixels! (No downward resampling here).

My apologies for the large size.

Details: Both shot in RAW (not me! :)) at 35mm f/5.6, 1/200 second, flash, handheld. Distance to subject: 1.6 m (about 5.25 feet). Converted to linear using LS342 using normal sharpening. Converted to JPEG at 7/12 (to speed up download). No other manipulation.

The first one was taken with the Tokina cheapie:
http://www.users.bigpond.com/ruda/demo/tokina19to35.jpg

...and this one was taken with the 28-135 IS (IS turned OFF):
http://www.users.bigpond.com/ruda/demo/canon28to135is.jpg

As you can see, the Canon lens has higher contrast and is slightly sharper. On the other hand, the Tokina is NOWHERE NEAR as soft as your Sigma (which was apparently a lot more expensive). And to be perfectly honest, I'm not sure if the Tokina was focused spot-on, bust seeing that I used the same area to focus on, let's call them as they fall... :)

If you'd like me to put both on a tripod and do a more controlled test, let me know. In the meantime, it would be interesting to see a photo taken with your Sigma of some large and small text at about 5 to 6 feet (same distance that I just did - sorry, I couldn't find a battery. :D).

[oops]

Rudi,

Great job! You are so reliable; wanna job?:)

This is an interesting comparison and the Tokina looks very good at this range. I don't think the tripod would make a large difference here with 1/200 sec unless other readers would like to see it.

I will bang out a text @ 5 to 6 feet tomorrow and try to get it posted tomorrow night.

Those of you who are happy with the Sigma 17-35 EX please don't be shy on this. I would love to see how this lens SHOULD perform as there are several happy owners on this forum.

For the record, I paid $616 for the Sigma. It was high compared to B&H but small businesses need to survive and most offer excellent service after the sale. I figure I paid almost 40% more than I could have so, if added to the $616, now I'm in the "L" range. (At least for my dream lens, the 28-70L.)

Stand by for the next shots and hopefully some other folks will jump in as well.

Chris

[Rudi]

Oops wrote:
I don't think the tripod would make a large difference here with 1/200 sec unless other readers would like to see it.


Chris,

I don't think a tripod would make much difference re: camera movement, but I still have a sneaking suspicion that *I* might have moved slightly after focusing with the Tokina (after the focus locked, I might have moved slightly).

All-in-all, I think the Tokina is bound to be softer than the Canon (because of the different focal range and the difficulty in making a wide-angle zoom such as that in the 35mm format), so the results above are probably very close to the truth. Let's not forget that there are people on this and other forums that think that the 16-35 L (and some with the 28-70 L) is soft compared to their other lenses!

As to the job offer, only if I can work my own hours, have a company house, a car, the just-about-to-be-released 1D replacement, and a willing assistant (Fern??). :D

As my bonus, after my first year, I'd like all my lenses to be upgraded to the "L" versions...

[Rudi]

OK folks,

Just to further clear up (confuse?) the issue, I have shot some newsprint to test all my lenses. (Something I could never see myself ever doing, but there you go... :)).

You can view the results of my test here:

http://www.users.bigpond.com/ruda/demo/lenstest/lenstest.htm

As I determined before, my Tokina 19-35 is slightly softer and less contrasty than my 28-135 IS at 35mm. Not bad though, considering the difference in price.

I tested all of my lenses. Not sure about how scientific the test is, or whether it is of any use to anyone here, but please have a look and let me know your thoughts...

[roine]

I have tested 70-200L2.8, 35-350L and 100-400L IS

I have done a low light test on my L lenses indoors.


The test can be reviewed at my web, http://www.roine.nu and the raw files can be downloaded at http://www.roine.nu/lenstest.zip or at a link near bottom of the page (under headline Bildspel)

The camera is a Canon D30.



Due two the debate on all sites I have compared them by using only my model light on my two Elinchrom 500 studio flashes against a newspaper. That to give some low light conditions.

I am aware that this will give some long expose time, eg up to several seconds. Se EXINFO at my web under each picture.

The first I saw was the wb issue, I used auto WB.

70-200L/2.8

----------------------------------------------------------------------------------------------------

zoomed too about 150 mm
CRW_9816 70-200L/2.8 at 2.8
CRW_9817 8
CRW_9818 16
CRW_9819 32

CRW_9820 100-400L IS at 5
CRW_9821 8
CRW_9822 32

CRW_9823 35-135L at 5
CRW_9824 8
CRW_9825 32

-----------------------------------------------------------------------------------------------------

Zoomed at about 350mm
CRW_9826 70-200 with Solignor 1.7x teleconverter (showed 2.8)
CRW_9827 8
CRW_9828 32

CRW_9829 100-400 at 5.6
CRW_9830 at 8
CRW_9831 at 32

CRW_9832 35-350 at 5.6
CRW_9833 8
CRW_9834 32


The jpg files are processed in PS with auto contrast and unsharpen (150, 1, 0)


There are a summary jpg, raw from the camera, side by side, for the two zoom range at the bottom of the page.


--
roine@roine.nu / http://www.roine.nu

[oops]

Rudi wrote:

As to the job offer, only if I can work my own hours, have a company house, a car, the just-about-to-be-released 1D replacement, and a willing assistant (Fern??). :D

As my bonus, after my first year, I'd like all my lenses to be upgraded to the "L" versions...


Sooo, what kinda *car* we talk'n 'bout? I drive a Dodge Cummins but I suspect this ain't yo-cup-o-tea.:)

My oldest son had his 29th birthday last night and I had to join the activities. I still burned the steaks while preparing these and got "the look" from the missus. I did the right thing: Log off, Smile, don't go back on-line for 24 hours.8) Works for me.

First, the 28-135 IS- same specs as yours:
http://www.pbase.com/image/3576155.jpg

And, the Sigma 17-35:
http://www.pbase.com/image/3576163.jpg

And, a closer look:

28-135 IS
http://www.pbase.com/image/3576167.jpg

17-35 Sigma:
http://www.pbase.com/image/3576173.jpg

The 17-35 is a much *brighter* lens than the Canon. I found that, for a good center loaded histogram, I could have done a plus one stop on the Canon and a minus one on the Sigma. Funny how the two lenses seem so far apart with their responses to my D30.

Roine and Rudi, very nice job on the lens demo. I have saved both for future reference. You both have some fantastic lenses and they perform as I would have expected. Any surprises for you?

Rudi, I thought you had the 70-200 f/4? The 2.8 is my other "dream lens".

[Rudi]

Oops wrote:
Rudi, I thought you had the 70-200 f/4? The 2.8 is my other "dream lens".

Chris,

I *had* the f/4 version, but it had some problems, if you recall (not optical!), and I couldn't replace it quickly enough, so I caved in and paid the difference! :) I have the non-IS model. The IS was worth an extra "new 85 f/1.8", and makes the lens even heavier, so I decided against it.

So far, no regrets, only when I have to carry it anywhere! :D

I got myself a pre-loved 300 f/4 L (non IS) this week, and this will have to do me for a while... (my dream lens is the 300 f/2.8 L IS, but I will have to wait until I can justify the expense to *myself*).

From looking at your latest photos (for which you got in trouble :)), your Sigma is MUCH softer that your 28-135 IS. What distance were the first two shots taken at? The small text is almost non-existent with the Sigma, yet easily read with the 28-135 IS...

I was kinda disappointed when looking at my test photos taken with the Tokina. I was surprised that it was that much softer than my 28-135 IS, because I can't honestly see that in my photos, but your Sigma is much softer again! I think you're doing the right thing by returning it!

Have you decided on a lens yet? Personally, if you already budgeted for it, I'd go for the 16-35 L... that is, if wide-angle is very important to you. If you can live with the quality of the Tokina (so far I've been able to :)), and you're lusting after the 28-70 f/2.8 L (as I do), then that's probably what I would do. The only thing with the 28-70 L is that it's a little short at the long end for my liking... (WHY can't we have everything???)

Good luck in choosing. Let us all know what you decide!

[oops]

I *had* the f/4 version, but it had some problems, if you recall (not optical!), and I couldn't replace it quickly enough, so I caved in and paid the difference! :)

Ah, yes. I remember the dust speck in the f/4. You made the right move. Remember the thread recently about Canon's top 3 lenses? I listed each lens mentioned and the number of votes for each. The 70-200 f/2.8 was the huge favorite with 10 votes.


From looking at your latest photos (for which you got in trouble :)), your Sigma is MUCH softer that your 28-135 IS. What distance were the first two shots taken at? The small text is almost non-existent with the Sigma, yet easily read with the 28-135 IS...

The first two were about 5.5 feet. The close-ups were about two feet. The 17-35 image quality does seem to fade with distance, which comes as no surprise.


Have you decided on a lens yet? Personally, if you already budgeted for it, I'd go for the 16-35 L... that is, if wide-angle is very important to you. If you can live with the quality of the Tokina (so far I've been able to :)), and you're lusting after the 28-70 f/2.8 L (as I do), then that's probably what I would do.

I returned the Sigma 17-35 and ordered both the Tokina 19-35 and the 28-70L from B&H today. I will not be using the 19mm very often, it's just important to address the wide end for special shots.

My greatest need has been a high quality lens that lives on my camera most of the time and the 28-70L has filled that role for hundreds of photographers with far more discerning eyes than mine.

This thread has been fun! It is why I love this place so much. Buying decisions are much easier when done this way. Thanks for the help!

[Rudi]

oops wrote:
Ah, yes. I remember the dust speck in the f/4. You made the right move.


Chris,

there was more to it than just the dust speck, the mounting ring was also damaged (I think the lens was made on a Monday), and that was the last straw! I was willing to let the dust speck go, and maybe have it repaired under warranty later, but the fact that it was an "L" lens, and was not put together properly got to me in the end, and I decided not to let Canon get away with sloppy workmanship...


I returned the Sigma 17-35 and ordered both the Tokina 19-35 and the 28-70L from B&H today. I will not be using the 19mm very often, it's just important to address the wide end for special shots.


Let me know how you like the Tokina, and *definitely* let me know about your 28-70 L! It's one that has been at the back of my mind for some time, and it would be good to put my mind at ease one way or another... :D


This thread has been fun! It is why I love this place so much. Buying decisions are much easier when done this way. Thanks for the help!


You're welcome! It's been fun for me, too... :)


P.S. Please note that my lens test page has been moved to http://www.users.bigpond.com/ruda/demo/lenstest/lenstest.htm because the free web server on which it was originally was becoming VERY unreliable (I've left the original where it was, but decided to put up the page on my website as well...)

[roine]

Hello oops

It was a suprice that 70-200L with the ceep 1.7x Converter is so sharp.

Roine

[schlepper]

If you're interested in standardized and objective information, try www.photodo.com. They have MTF charts and detailed technical info for pretty much every lens on the market. It's my first stop when considering purchase of any lens.

You'll see that the Canon 28-70 is rated significantly higher than that Sigma.

[oops]

roine wrote:
Hello oops

It was a suprice that 70-200L with the ceep 1.7x Converter is so sharp.

Roine



Actually, that is a Rudi set up, but I agree. Give the credit to the 70-200L, not the 1.7x.:)

[oops]

schlepper wrote:
If you're interested in standardized and objective information, try www.photodo.com. They have MTF charts and detailed technical info for pretty much every lens on the market. It's my first stop when considering purchase of any lens.

You'll see that the Canon 28-70 is rated significantly higher than that Sigma.


Thanks, I go there often. I must admit that I don't understand the graphs even after reading theirs and other explainations. I do love the grade system, even though sometimes those I think should be very high are low and visa-versa.

I also like the lens reviews from users @ http://www.photographyreview.com/reviewscrx.aspx

Chris

[Rudi]

Great minds think alike! :D

I go to www.photographyreview.com as well, and actually prefer to read through reports of actual users, rather than just compare the raw MTF graphs on www.photodo.com. There are things that MTF graphs won't tell you, IMO...

That said, I *always* double-check any potential purchase with photodo, just to make sure! :)

[Rudi]

Oops wrote:
Actually, that is a Rudi set up, but I agree. Give the credit to the 70-200L, not the 1.7x.:)

Chris,

I think Roine is actually referring to his setup.

One thing I can definitely state is that the Canon EF 1.4x Extender is NOT cheap! :D

[oops]

[Chris,

I think Roine is actually referring to his setup.

One thing I can definitely state is that the Canon EF 1.4x Extender is NOT cheap! :D

Oops. I got confused on that one.:) He was responding to my "any surprises" question. Sorry for the dim lights, Roine. How long was I out?

The Canon 1.4x has been out of my range for as little as I would use it at this point. I did buy the Tamron 1.4x and it does a very respectable job for now.

[Rudi]

oops wrote:
Oops. I got confused on that one.:)

LOL! :)

Quote of the week!

Chris, that one cracked me up! Just the combination of your user name and then the comment was priceless! :)

[Rudi]

Hi everyone,

In my quest for sharper and better lenses, here's an update of what I've been up to:

After running too late to pick it up on Saturday afternoon (the shop closes at 2PM), I picked up my brand new 20mm f/2.8 plus lens hood yesterday.

Damn! What a nice lens! :D

I could get used to this "prime" feeling! After using my new 28mm f/2.8 for most of the weekend (bought it last week), and now this new toy for a few hours, I can see that this will be addictive!

I have taken my 28-135 IS out of the camera bag, so as not to divert my attention from my other lens choices in that focal range (20/2.8, 28/2.8, 50/1.4 and the short end of the 70-200 f/2.8L), and I don't miss it at all! I have also just sold my Tokina 19-35mm zoom, and while I will miss it a little (it did such a good job for so little money), I always used it on the wide end, so the 20mm prime will replace it rather nicely. Plus, I found it a good home, even if it is in the colder climates. :)

So, keep your eyes peeled for an update of my "Rudi's lenses tested" page in the next few days:

http://www.users.bigpond.com/ruda/demo/lenstest/lenstest.htm

I will also post some new shots in the "critique corner" forum as soon as I get the chance.

Anyone want to offer me any money for a mint 28-135 IS with all original documentation and packaging? (I have had an offer of $325 so far...)