If your shutter speed is over 1/500 (or so) then IS will not help in any mode as the IS element cannot move fast enough. Also (confirmed with a Canon Rep last Thursday) IS can interfere with AF acquisition and tracking as I have found and posted. The latest IS systems, like the 100-400 Mk2, are much better in this respect but it can still cause problems.
At 1/3200 IS is a waste of space and you risk focus problems as you have found out. However when you simply can't get the required shutter speed at sensible ISO then IS comes into it's own and many find it very helpful.
For what it is worth I find my 100-400 Mk2 extremely difficult to hold steady so, at 1/400 or less the IS is turned on in Mode 3. Mode 3 focuses first then activates the IS and works much better for me and others I know. I simply don't bother with Modes 1 and 2 as they interfere too much but Mode 3 works! If, unlike me, you can hold the 100-400 reasonably steady then just turn the IS off and forget about it. I need IS on my 100-400 Mk2 but it is simply not used on my canon 800mm unless I have to go below 1/160 hand held (not happened yet) - as I said I simply cannot hold this lens steady reliably.
Life is for living, cameras are to capture it (one day I will learn how!).