First, there are many more zooms in circulation than primes, so the odds of somebody finding one that isn't as good is much greater.
Zooms have more moving parts, meaning more potential areas where optical alignment can be slightly off. Generally, primes only have moveable focusing groups (IS lens groups move as well, but that trait is common to both types of lenses).
Zooms, in general, are a compromise. They have to perform the work of several different prime focal lengths, sometimes with drastically different optical requirements. A wide-angle lens with a focal length shorter than that of the physical lens requires a much different optical correction than does a short telephoto with a focal length longer than the physical lens should provide. Put both of those requirements into a single zoom and you've got a very complicated package.