Factors to include when comparing lenses?

For example:
http://www.the-digital-picture.com …&SampleComp=0&F​LI=3&API=2

Should that link not work correctly, comparing 100% crops of an EF-S 17-55/2.8 IS USM to an EF-S 18-55 3.5-5.6 IS, both on crop sensor frames (50D), both @ 35mm/f5.6 for an "equalesque" comparison.

Now, in my mind, which may be terribly wrong, I'm not seeing a vast difference there. Not enough to generate the separation in cost. What I've taken into a moderate account is the lack of sharpness for the 17@2.8. I know that there has to be a vast quantity of other variables/consideratio​ns that I need to be concerned with which leads me to the grand title up top, what should I be considering other than cropped IQ? To what degree does better CA handing outcredit a softer image?

Have read the wonderful stickies; namely:
https://photography-on-the.net/forum/showthre​ad.php?t=97056

But I'm encountering difficulty when attempting to track down comparisons that measure CA for a wider selection of lens. If distortion is one of the larger issues with one lens over the other, should I neglect that keeping in mind that correction can be made PP?

It's not always just about image quality. The 17-55 is a constant f2.8 across it's range, and you pay for that.

There is no one answer, imo.

A better lens for one might not be the same for another, in the same focal length or range.

If you shoot panoramic landscapes, edge sharpness is very important, if you shoot portraits, you might actually want softer edges.

Someone might absolutely need faster glass so can tolerate lesser performance in other areas when compared to another lens.

Then there is the "look". Bokeh, contrast, color, microcontrast. These are not equally important to everyone.

If you are paying a premium for a lens and not using it where it can display it's strengths, then, for you, that quality is not important, or shouldn't be, in your buying decision.

I've said it before and I'll say it again: AUTO FOCUS, AUTO FOCUS, AUTO FOCUS!

Is it fast enough, accurate enough and good enough in low light for your needs?

One thing about lens choices that is very important to me is that ALL of the lenses for any particular camera have controls that are as similar as possible. I want all of the focusing (and zoom rings for zoom lenses) to turn in the same direction for the same change. I want all of the zoom rings to be on the same side of the focusing rings (which one is closer to the camera). I want all of the focus and zoom control rings to be the same style and feel approximately the same to the touch.

The reason for these rules is so that my hands don't have to re-learn anything when I change lenses on my camera. I cannot afford the time delay or cause errors in my work flow to occur.

Josepi wrote in post #9962869
Now, in my mind, which may be terribly wrong, I'm not seeing a vast difference there.

Not a big difference at all @ f/5.6, but a huge difference @ f/2.8, wouldn't you say?

itzcryptic wrote in post #9964515
Not a big difference at all @ f/5.6, but a huge difference @ f/2.8, wouldn't you say?

Of course

Too many variables to consider whilst shopping around, continually upselling myself into something outrageous in price, then whittling it down with what I'll consider my actual uses to be.

Everyone else, valid points. Just want to make sure I'm covering all my bases.