Why don't we see what Canon is up to?

[cactusclay]

I just had a thought, due to my recent experience with lens sample variation. Why don't a few of us take a tripod, clip board and a page out of a phone book, into a local camera store and try out some lenses. If people were interested, they could volenteer for a specific focal length and try out three or four lenses in that focal length and see what sort of results they came up with, then at least we could all have an idea about how much sample variation there really is out there, instead of not really knowing for sure if people reporting bad copies, were just a little off their rocker, or they were telling the truth. The worst that could happen is Canon might catch wind of it and tighten up their quality control. Any takers? :lol:

[Tom W]

Not quite what you're asking for, but you can peruse the output of various lenses and such at PBase. Granted, they aren't all full-sized images, and some images are poorly concieved, but there are samples of many cameras and lenses in the database.

Here's a link (scroll down for the lenses):

http://www.pbase.com/cameras/canon

[robertwgross]

I have no problems with Canon quality control.

I cannot make the same statement about some other vendors.

---Bob Gross---

[Redbird_xo]

If the sample isn't statistically drawn, it won't represent the population. Therefore,the test results won't be fair thus won't be useful.

[raylks]

How much samples are needed for the statistics? And did we suffer from R (reproducibility) & R (repeatibility) issue in such remote test?

[Mike H]

How much samples are needed for the statistics? And did we suffer from R (reproducibility) & R (repeatibility) issue in such remote test?

That depends on what kind of test you set up. To get all of the properties that one normally enjoys using statistics for hypothesis testing (normally distributed sample means, for example) a test has to be carefully conceived. One thing you need is for the test to be truly random, as someone pointed out in a prior post. You would want to draw samples from a number of different sources. Why? Consider what would happen if there were one bad batch of lenses out of several hundred batches, but you only drew samples from that one bad batch (say they all went to the same camera store). You would definitely draw an incorrect conclusion about the population of Canon lenses.

But to answer your question in the simplest terms, the stats boys usually say that you need at least a sample of thirty if you want to do anything worthwhile.

To me the tough thing is determining the criteria of acceptability. For instance, if you had a way to calculate the type of stats that you see at the photodo.com site (modular transformation functions), which can be boiled down to a numeric term at each f stop, then you could establish what your standard is in a way that people could understand (and then agree or disagree with). For example, you could say that if the weighted MTF score for a lens was below 4.0 it doesn't meet your standard. Then you could establish the probability of drawing a lens at random that would or wouldn't meet you standard.

It would be a LOT of work to do a fair test. ;-)

Mike

[robertwgross]

On top of what Mike stated, let me add that a proper survey does not have participants that are self-selected. A proper survey either uses 100% or else randomly-selected participants.

---Bob Gross---