Tips for doing your own "Lens Comparison"

Been struggling with a little upgrade of lenses, and because of it I have read many reviews, seen infinite sample photos, and even comparison images that claim towards one lens or against it, without so much as a clear convention of rules.

For those of you who like to compare your new lens side by side with the current one to see if you made the right purchase, how do we establish some common sense rules?

For instance, from the Lens Sample threads, I learned that two types of photographs should be used for the comparison:

* Uncropped version (personally I prefer without PP)
* 100% crop, which tends to show some details of sharpness, CA, Vignetting, etc.

For the purpose of this thread, lets use the following:

Suppose that for some reason, you want to compare the Canon 70-200 IS f4 with the Canon 70-300 f4-5.6 IS.

I will post below what I would do to compare. Please teach me !!

(REMEMBER, WHAT MATTERS IS THE PROCESS, NOT WHICH LENSES WE ARE COMPARING, DONT TELL ME L IS BETTER BECAUSE I KNOW THAT)

OK, so I would gather full and %100 pics of:

- Both lenses at the widest common focal length (if 55-250 vs 70-200, then 70mm being widest)
* And both lenses wide with f/stop open to the max common aperture
* And both lenses wide with f/stopped down to a well known range (f11 seems to be widely accepted without incurring into diffraction blurriness)

- Then, the same but for both lenses at the longest common focal angle, at open and closed apertures.

Questions I got:

- What are some ideal test compositions? For instance, do we want to try a hillside a few hundred yards away, or a subject half way between ourselves and whatever background is there?

- What is a good method of testing an IS vs non IS lens?

- Do you shoot in M at the same values for both lenses? Green zone?

(I'll stop my questions there... for now)

I imagine several thick books can be written on this topic.....and a few dozen PhD thesis.
There are so many variables, you can't possibly do justice to many of them at once, so the fall back position is to not try to be comprehensive, but choose image parameters that are important to your style of shooting. Devise a test that is under conditions and variable ranges that reflects what you might use for a given lens.
Perhaps if you are testing a 400mm lens you might often shoot "a hillside a few hundred yards away, or a subject half way between ourselves and whatever background is there." One gripe I have with some of the chart style tests is that they are all shot of a flat surface at very close range, very close to MFD, and this may not be very relevant to the use of a given lens. Foreseeing potential variables and controlling them is the stuff of good experimental design.

For reproducibility sake I always use a tripod and Live View Silent Shooting and a remote to minimize vibration. This also allows manual focusing at 10X, but then perhaps you are testing AF calibration? I usually do lens tests in Av and vary it from wide open to before the onset of diffraction (f11 or so).

You have to decide what is important to you and control variables that may impact those results.
Mike K

i say you should test it somewhat like you plan to use it...i can't tell you how many times i've seen people taking test shots of long telephotos indoors under poor light...for the most part i think getting outside is a good idea...tripod mounted, IS off, same settings, just swap the lenses in and out...for testing between a 70-200, and 70-300 i'd probably take a shot at 300mm with the longer lens...and then one at 200mm and crop it to the same composition to see if there's a benefit to the longer lens...but if you're testing a longer lens...set up a box with some writing outside at a decent distance, and then shoot that..

• Compare common f/stops.
  
• Compare common FL. Keep in mind that the rated FL may not match, so shoot for 'same frame content' so you can compare detail at the same size.
  
• Use a camera mounted on a tripod.
  
• Use a remote release, or a self timer, to eliminate vibration caused by you touching the camera.
  
• Where one lens has larger aperture or longer FL, don't bother...there is nothing to directly compare.
  
• Try to have a good target with some fine details that fall at the center and at the edges of the frame.
  
• Avoid shooting in windy conditions, as the wind might cause blur at the subject (wind blown hair, for example) and thus ruin any truly direct comparison.
  
• Don't necessary compare at MFD...compare at more typical shooting distances.
  

Awesome tips so far!...