How Do Canon Test Lenses? Model consistency/Canon QC?

There seems to be a lot of posts here and elsewhere on faulty Canon lenses. Subjectivity and errors in testing technique may account for the bulk of these but there still seems to be significantly more genuinely faulty Canon lenses than what we hear from Nikon users.

Anyone know how Canon tests each new lens before it leaves the factory? What do they do before a "QC passed" stamp is placed on the lens?

I take it the lens manufacturing process is well automated and raw materials are standardised. The lens is designed, raw materials sourced and tested, machines retooled, manufacturing process identified etc. So what can go wrong such that one lens turns out good and another turns out bad?

And how does QC testing at manufacture differ to the QC at Canon service centers when they're reparing a lens? Why do we so often hear that lenses come back worse (or with different problems) to when they went in?

I hate to knock the brand, I've always been a fan of Canon and their photog products. But it seems their QC displays a apathetic view of the hobbyist consumer. Hey they're not exactly flush with competitors and they know the cost to jump ship is huge given our huge investment in lenses and bodies equipment.

This is only my opinion, but I'm looking at photozone.de as they test more and more Nikon lenses and optically, they seem better to be. A slight amount but better, considering the 80-200 vs 70-200L, the 50mm f1.8's, 85mm f1.8's, 80-400 vs 100-400L, and so on.

Anyway in terms of QC, most of the reported errors are due to the tester in my view, having seen some shot and methodology, or the lack there of. But I think some really are bad. Photozone.de had to retest the 300IS prime, 70-200f4L, and a few other Canon's if I remember and got significantly better numbers/resolution. I can't tell you why, but I hear that lenses are hand assembled, so if that's true, maybe some Canon lens employees might have to lay off the Sake or something.

PhotoZone specifically warns you about comparing test results between different "systems".

PhotoZone states:

"Nikon (APS-C DSLR) LENS TEST REPORTS / REVIEWS - Results not cross-system comparable"

and

"Canon EOS (APS-C DSLR) LENS TEST REPORTS / REVIEWS - Results not cross-system comparable"

So the "slight amount but better" differences you cite, may, or may not, be due to differences in lens quality but due to inherent system differences (such as differences in sensor pixel density, differences in the aggressiveness of the anti-alias filter, differences in inherent in-camera image processing, or other factors).

Enjoy! Lon

I know that an MTF value of let's say 1850 on a Canon is considered "excellent" but it requires MTF value of 1950 on the Nikon to be classified as "excellent" due to the different resolving ability of the sensor. But for example, on the Nikon 80-200, the lens has excellent class sharpness at center at f2.8 for all focal lengths tested, while Canon 70-200L f2.8 does not. At some focal lengths wide open, it rates very good.
I'm assuming that the resolution category of excellent is visibly noticable over very good. So I'm comparing not raw numbers but trends and class of resolution that the lens falls under specific focal lengths and aperatures.

But I still think the Canon lens assemblers should lay off the Sake.

There will all always be areas where Nikons can be considered superior to Canon and visa versa.

But what I find more concerning with Canon is that they seem to have more lenses that don't perform at the level of their specs or how the published MTF charts suggest they should work. I agree with you Tee Why in that a lot of this could be put down to user testing errors. Regardless I think when people post here with QC issues there are enough posts to point the original poster in the right direction when it comes to testing lenses correctly and/or what is considered acceptable for that lens.

I didn't know that Canon lenses were hand assembled though. That has potential to introduce errors in the manufacturing process. But surely the main Canon factory should have some sort of standardised testing to ensure focus points can be accurately achieved, edge to sharpness/contrast/res​olving power etc across apertures equates to the MTF charts? And I would hope that individual service centres are equiped with the same level of testing?

The main areas that have concerned me is the flare issue with the 24-105L and the portrait mode softness with the 70-300IS. These lenses were released last August and both had some design flaws, so I'm not sure if it's the design and/or the assembly. Sharpness aside, I really don't know if Canon has worse QC than anyone else though. I don't know if anyone keeps that kind of data. I can say I've never returned a lens due to defects, although my lensbaby has poor corner sharpness and the AF doesn't work....

At least Canon owned up for the 24-105 flare issues quick smart which is a good sign But it's a shame that it wasn't until consumers told them of the problem that they realised.

I suspect the key here might be that Canon is sticking "IS" in more lenses. "IS" adds enormously to the complexity of the lens. This additional complexity inevitably leads to a grander challenge in both design and manufacture.

Enjoy! Lon

It's highly unlikely that they test every lens that comes off the line, it would be physically impossible.

Whilst a lens is a precision item, so is a car and both are mass-produced. Bad ones pop up from time to time.

C'est la vie

Probably a lens is given a quick look-over befor packaging, but they probably only shoot test photos from one sample out of,say, 500 for EF series lenses and one out of 50 for L series lenses. What really matters is how tight the manufacturing tolerances are and how much effort they expend to keep up to them.

GSH wrote:
It's highly unlikely that they test every lens that comes off the line, it would be physically impossible.

Whilst a lens is a precision item, so is a car and both are mass-produced. Bad ones pop up from time to time.

C'est la vie

It is highly likely that every lens, especially the most expensive ones, are tested in some manner after they are manufactured. And, most likely right on the assembly line with some sort of computer driven test fixture.

What makes you think this would be "physically impossible"? Do you have experience in the production of "reasonable volume" high precision optical devices?

These lenses are not rolling off the assembly line like ball point pens.

Enjoy! Lon

I've never been in a manaufacturing environment so really have no clue but would have logically thought it could be done as part of the assembly line process as londuck says. As with the car analogy each car is delivered with delivery miles. Of say 20 delivery miles I would guestimate that less than 1 mile would be actually transporting from factory to the showroom (i.e. loading from factory to truck, truck to ship, ship back to truck, truck to showroom), the rest would be testing miles.

Anyway it'd be intresting to get an insider's view from Canon as to what happens inside the land of Oz

You can't compare these sort of tests across camera platform because of the differences in pixel pitch and anti-alias filter design, I think photozone even have a warning posted about that.

My experience is of the electronics inductry but I expect it follows across. Production line time is money, in comercial products like mobile phones a 10 second line test time is considered long. I expect they do some masic electromechanical test (check the AF and iris actuators) and relly on statistics, there may be a brief visual inspection or possibly a simple laser and sensor check that it actually has some glass in and is not grossly missaligned.

Bear in mind also that a mobile phone is many order of magnitude more complex than a lens.

That's what I was thinking. My MP3 player, PC, household electronics etc come through with a tested stamp. Something as simple as a lens, in terms of its working function, could be easily tested beyond a simple visual once over to ensure common areas are checked - flaring, focus hits the assigned AF point(s), contrast, resolving power, edge to edge sharpness etc.

For a zoom I'd think it's a simple case of mounting it to each of the current model bodies (testing area/subject/charts already setup), and for each level of zoom testing at various apertures across the range. The review would be as simple as pressing a button to ensure the shots equate to how the lens is designed to work and would rely on computer algorithims rather than human checking. In fact the whole process from mounting to various bodies, to taking the test shots to the review could possibly be done with little to no human intervention.

At the end of the day a lens is physics and in a controlled testing environment there is no reason why one lens should perform differently to another.

Goodness, they wouldn't do it like that, it would computer controlled testing with custom jigs to interface to the lens elecronics and possibly optics, all automatic, possibly some human help in putting the test item in and out of the fixture. I would expect a test time of seconds, with a simple go, no-go indication. The no go's are probly dumped into a bin for debug. It is normal for production test limits to be quite loose, often just a functional rather than quantative test, so I would not expect this to be an effective screen of all but the worst lenses.

I am sure Nikon and other mass production manufactures run a similar process. It may be that much more expensive optical manuafctures test more tightly for performance, but that explains the price difference.