There is no point in comparing a 35mm or even 100mm to 400mm. I mean, there is, but you can't expect the same quality. This is not necessarily a prime vs zoom problem. I'm sure the new 100-400L II has roughly the same IQ as a 400mm 5.6L at 400mm, provided they're both good copies. And it's very likely that they will both fail when comparing them to the 100mm or 35mm shots taken at 1/4 or 1/12 distance.
When you shoot 400mm (and higher), you obviously shooting something at a distance. Unless you do it in a vacuum, you need to understand the factors that negatively impact IQ when shooting with a longer lens outdoors.
It's easy to see if you set up your lens on a tripod, use live view, focus on something at a greater distance (stop sign, telephone post, etc.), use 10x magnification, then watch the IQ vary in real time on the LCD. Heat currents, haze, etc. all impact the IQ. Quality will be going in and out constantly.
Now, if you do that with a 35mm lens, you won't see any of that. Even if you take a shot of the same subject (stop sign, telephone pole) at the same distance, everything will be nice and sharp even at 100%. Magnification from a 35mm lens is not going to make those heat currents visible.
Another easy way to see this phenomenon is to say pick a 600mm f/4L II prime and compare it with a zoom, say EFS 15-85mm used at 35mm. No doubt that the 600L is by far the superior lens. BUT if you do the same test as mentioned above, I'm sure that most days the sharpness of the 15-85 at 35mm will beat the 600L when you take a photo of a far away subject.
Basically it's the same phenomenon as when we compare how a nearby mountain looks vs a far away mountain. The far mountain looks hazy, and we just basically magnify that haze with our super tele lenses. That's actually why the Moon astronauts had an extremely hard time judging distances on the Moon. Humans are used to estimate distances based on how far away objects look on Earth due to haze. The Moon doesn't have that "aid".
Now, if you take a shot at 1/4 of the distance with the 100mm lens, or 1/12th of the distance with the 35mm lens, that helps these prime lenses even more vs the 100-400L IS II. Some respected lens review sites such at SLRgear.com and The Digital Picture actually do mention sometimes that while testing a long lens, they're shooting through more air which impacts the sharpness/contrast results negatively (it's very very minor though inside their studios but it's true.)
What you can do is test the 3 lenses inside using a powerful flash. Set up a test subject 15-25 feet away, use a tripod, flash then fire away. You'll see that the 100-400L will yield excellent sharpness and contrast under these conditions. Weather, heat, etc is not working against the lens in these conditions.