How reliable are online lens comparison tools

Hi,

I want to compare lenses side by side at image quality, to avoid paying money for a lens that not worth it. I found "the-digital-picture.com" having such a tool and I can see differences between lenses. But the problem is that some well reviewed (by owners) lenses having very bad images when you compare some other similar or cheaper options. Must I rely such websites or I must go with reviews of owners?

Thanks!

chaturanga wrote in post #17644578
Hi,

I want to compare lenses side by side at image quality, to avoid paying money for a lens that not worth it. I found "the-digital-picture.com" having such a tool and I can see differences between lenses. But the problem is that some well reviewed (by owners) lenses having very bad images when you compare some other similar or cheaper options. Must I rely such websites or I must go with reviews of owners?

Thanks!

Heya,

It's a useful tool to give you an idea, but it also will make almost every lens looks "awful" at the same time. Measurements in general is a lot like pixel peeping, you'll just end up hating everything made.

I would look at reviews of features and general performance, but beyond that, with the grain of salt always present, look at sample images from users in the real world and not in lab measurement setting type things and see for yourself if it has qualities that you want. I trust the sample images threads here, and flickr pools focused on a lens, to see if it's something interesting to me, before worrying about a sharpness test on a chart.

Very best,

The problem is that there is variation in lens performance between samples and variation depending on which body a lens is mounted on. None of the lens test sites test a statistically significant sample size. Most will never test but only two and that is if there is something obviously wrong with the first one.

Best bet is to read as many different reviews and not take any one as gospel. Then make sure you test the lens on your body.

It is indeed confusing. Take the 18-55mm STM lens, for instance. Photozone.de found the sharpness and overall performance to be excellent. Yet other reviewers found it to be about the same as its 18-55mm predecessors. Sample variation could certainly be part of the reason. Maybe there are other reasons too - could the reviewers have biases? I don't know. Keep in mind that the reviewers are sales people. They make money when you buy (click through their web sites, actually). So expect some hype in the reviews, and possible minimization of issues. My opinion.

Be alert to the (small) possibility of a faulty lens. If the lens you buy is not right, return it within the return period for a replacement or refund.

Thanks for your reply, yes when I look those images showing distortion, chromatic abbreviation, lack of contrast and lack of sharpness through all ranges of many lenses, I am so much confused to buy which one. I decided Canon 55-250 STM for telephoto zoom lens, it's images seems more sharper and accurate in the web site I have mentioned above and also user experiences are very very good for that particular lens.

Yes you are right, and lots of time our eyes not dealing with so much pixel level details, overall satisfaction when looking at real user shots and reading user reviews may be better to decide. The best is if matches user reviews and technical reviews, for example Canon 55-250 STM seems such a great lens with a very nice price.

You are right, I am not someone always printing out photos, I will keep my shots on the digital media, and an overall quality of sharpness will be enough for me.

I will not zoom all of my photos and will not check if the corners of the photo has chromatic abbr. or there is a distortion to be fixed bla bla ) I will not make myself unhappy by zooming and zooming in to find inaccuracies that is my naked eyes did not care! If my eyes are happy with a shot, it's ok for me

And for sure, there are no camera body or no lens that will shoot perfect image. Perfect is our eyes not lenses!

chaturanga wrote in post #17645846
Perfect is our eyes not lenses!

You do realise that the consistency and "quality" of even most cheap lenses from the major manufacturers will be better that that of a cross section of human eyes. Most people's vision is far from perfect, it's just that for most people they never notice, it's only the optician at an eye exam that is likely to detect most deficiencies in vision. Not only that but human vision quality varies, over even quite short time periods, and with lots of factors, simply getting tired will negatively affect your vision for example.

Unfortunately this also then affects the way we see the images that we create. So beware!

Alan

Two things. I just acquired an 18-55 STM. I had the IS II. The STM is better optically, build-wise, and AF accuracy. It also does 1:1 macro with a 25mm ET.

When perusing review sites, please remember that any reviewer, for a huge range of reasons, can post shots worse than a lens is capable of. It is not possible to post shots better than a lens is capable of. I generally like Bryan's (TDP) take on hardware, but he has posted some samples that look like crap compared to my results with the same lens.

GeoKras1989 wrote in post #17645917
When perusing review sites, please remember that any reviewer, for a huge range of reasons, can post shots worse than a lens is capable of. It is not possible to post shots better than a lens is capable of.

That is a very good point. Certainly if you saw more than a handful of positive reviews for a lens you should be reasonably confident it's good (allowing for sample variation).

One strong example of that is the Samyang/Bower/Rokinon 14mm (known for iffy qualify control); as a non-scientific guess I'd say probably 8 out of 10 reviews say it's great, 1 says it's bad, and 1 says it's bad then gets a replacement and says it's great. My first one I got had problems, the replacement was great.

I also think there are better copies of the Canon 50mm f/1.4 than I've got, and was never as positive of my copy of the Tamron 70-300 VC at 300mm (compared to many reviews) - even though I still think it's a great lens for the money.

If you can find one of Roger Cicala's (Lens Rentals) comparison blog articles about the lenses you're interested in then that'd be very useful as he has loads of samples to play with.

GeoKras1989 wrote in post #17645917
Two things. I just acquired an 18-55 STM. I had the IS II. The STM is better optically, build-wise, and AF accuracy. It also does 1:1 macro with a 25mm ET.

When perusing review sites, please remember that any reviewer, for a huge range of reasons, can post shots worse than a lens is capable of. It is not possible to post shots better than a lens is capable of. I generally like Bryan's (TDP) take on hardware, but he has posted some samples that look like crap compared to my results with the same lens.

Exactly. Something else to keep in mind, is most of the photographs that people will say are stunning, amazing, and thought provoking are generally because of the subject matter and perspective, not because of the IQ of the lens used. While no one is going to complain about having a higher quality IQ when they capture the once in a life time image, the IQ isn't nearly as important as people with GAS make it out to be.

sploo wrote in post #17645969
That is a very good point. Certainly if you saw more than a handful of positive reviews for a lens you should be reasonably confident it's good (allowing for sample variation).

One strong example of that is the Samyang/Bower/Rokinon 14mm (known for iffy qualify control); as a non-scientific guess I'd say probably 8 out of 10 reviews say it's great, 1 says it's bad, and 1 says it's bad then gets a replacement and says it's great. My first one I got had problems, the replacement was great.

I also think there are better copies of the Canon 50mm f/1.4 than I've got, and was never as positive of my copy of the Tamron 70-300 VC at 300mm (compared to many reviews) - even though I still think it's a great lens for the money.

If you can find one of Roger Cicala's (Lens Rentals) comparison blog articles about the lenses you're interested in then that'd be very useful as he has loads of samples to play with.

Oh, yeah. Copy variation is real. I've tried 3 copies of the 50 1.8 II. None of them would focus. I have a 50 1.4 that is excellent at f/1.4. My first copy of the 85 1.8 needed to be at f/2.5. Current copy is dead-on, on both my bodies. (Sorry, I have no way to pixel peep the Elan 7.)

vengence wrote in post #17646028
Exactly. Something else to keep in mind, is most of the photographs that people will say are stunning, amazing, and thought provoking are generally because of the subject matter and perspective, not because of the IQ of the lens used. While no one is going to complain about having a higher quality IQ when they capture the once in a life time image, the IQ isn't nearly as important as people with GAS make it out to be.

When I go shopping for lenses, I consider:
Focal length -> max aperture -> IS -> size & weight -> cost -> reviewed IQ, in that order.

So many factors affect IQ that very few images I make even come close to what my gear is capable of. Want perfection? Shoot in a studio where you control EVERYTHING.
I bought a 100-400L because my 70-300 IS USM sucked at 300mm. I didn't need a reviewer to tell me that. I could see it for myself.

I never trust user reviews. They are subjective and written by someone who has made the decision to purchase a particular lens so their reviews tend to reinforce their decision. Who's going to spend $$$ on a lens and then write a review saying what a dumb choice it was. I also distrust sample images posted to a forum. Those most likely have been heavily edited and resized for display on a monitor and as such, most of the image data is gone. Only unedited raw samples are of any value.

That leaves review sited like the-digital-picture.com, DxO and others. The data they present may not fully show what the quality of a lens will be in every day shooting, but at least their process is defined and the same between lenses. They offer a reproducible and comparable data point.

But nothing beats first hand experience. I always try to borrow one and take a few pictures myself. When that's not possible, general consensus seems to work pretty well. There are lenses that have a solid reputation like the 100mm macro or the 70-200 f/2.8 II that many have used and agree that the IQ is exceptional.

GeoKras1989 wrote in post #17646078
(Sorry, I have no way to pixel peep the Elan 7.)

Come on man, try harder

gjl711 wrote in post #17646109
I never trust user reviews. They are subjective and written by someone who has made the decision to purchase a particular lens so their reviews tend to reinforce their decision. Who's going to spend $$$ on a lens and then write a review saying what a dumb choice it was. I also distrust sample images posted to a forum. Those most likely have been heavily edited and resized for display on a monitor and as such, most of the image data is gone. Only unedited raw samples are of any value.

That leaves review sited like the-digital-picture.com, DxO and others. The data they present may not fully show what the quality of a lens will be in every day shooting, but at least their process is defined and the same between lenses. They offer a reproducible and comparable data point.

But nothing beats first hand experience. I always try to borrow one and take a few pictures myself. When that's not possible, general consensus seems to work pretty well. There are lenses that have a solid reputation like the 100mm macro or the 70-200 f/2.8 II that many have used and agree that the IQ is exceptional.

Agreeing with your post, but of course it's worth noting that sample variation may well hinder "I always try to borrow one and take a few pictures myself".