I posted this question in one of the Sigma 17-70 f/2.8-4.0 threads, but I think it got buried away within that thread. Let me start a new one here...
Why don't lens manufacturers make all zoom lenses f/2.8 (without changing any of the optics - just mechanically allow for larger apertures), and let the user decide if the quality is acceptable or not when shot wide open? This would be similar in essence to camera manufacturers allowing ISOs in the 6400+ range - sure, the quality is horrible above that range, but at least users have the option of going there.
I know it's not physically easy or cheap to make any lens f/2.8 well. However, from what I understand, lenses with non-constant apertures simply have a chip inside the lens that electronically tells the camera "do not use an aperture larger than X when at focal length Y" (this might actually be done mechanically by the lens itself, but I doubt it).
So why can't camera manufacturers simply lower X (which is probably just an 8-bit integer stored in a look-up table and could easily be changed), and let the user decide if it's good enough quality or not?
I suppose there's an eventual limit to the opening of the diaphragm. However, take the case of the 18-55mm f/3.5-5.6 lens (D indicates diameter of the diaphragm):
18mm @ f/3.5; D = 5.1mm
18mm @ f/2.8; D = 6.4mm
55mm @ f/5.6; D = 9.8mm
55mm @ f/2.8; D = 19.6mm
So, unless I'm missing something, the kit lens could certainly do 18mm @ f/2.8 without requiring physical re-design. Note that I'm not saying the lens will do this well, but why shouldn't we have the option of using it, just like we can use ISO 12,800 if we want. A focal length of 55mm with f/2.8 might not be possible, depending on if the diaphragm can physically get that large.
If I had a enter a guess, I would probably say a) marketing/product differentiation, and b) a large class of beginner users would shoot in that range expecting great results and become very disappointed. So the next logical question is then, why don't they disguise these wider apertures as they do with ISOs (e.g. HI/LO settings)?