It really depends upon which cameras you are mixing...
7D and 5DII controls are laid out nearly identically, altho the 7D has a few more buttons and uses a different off/on switch. The two models work together pretty well, overall.
30D and 5D Classic are practically identical in layout and overall functionality.
Some other cameras are very different from each other.
And it can be a bit of a pain when cameras use dissimilar accessories... different batteries and chargers, different types of memory.
It's not the controls you use all the time that tend to be confusing, you can learn to switch between those on your cameras pretty easily for the large part. Nor is it a problem when you're shooting rather sedately.
The times I've had issues were when I needed to use something rarely used, such as top LCD illumination button. On 50D it's on one end of the array, on 5DII it's on the other end. Sometimes I'd have a pause in shooting, to stop and think about this sort of thing. It always concerned me, too, altho I never made a mistake with it, that the image review button and the image delete buttons are in the same location on 50D and 5DII.
I've also occasionally had problems when shooting very fast. I tend to change the settings on the second (and sometimes third) cameras to match, whenever I change the settings on the first camera, to have them ready for a quick swap. I also change some common settings while keeping my eye to the viewfinder. The controls have to be pretty much second nature to do that, and swapping dissimilar cameras makes that trickier.
Have I missed shots because of it? Probably. But not a lot and I can't really point to examples (since I missed them!)
It's not always just the control layout, either. Even when using two or three identical cameras, it's easy to overlook a memory card filling up or a battery running low in one or the other, until you go to shoot with it.
Oh, and when I occasionally go out and shoot with a film camera, I find myself staring dumbly at the back of it looking for a histogram more often than I'd care to admit. 