I would suggest just switching the IS mode to Mode 2; I use that one on the 100-400 a LOT and it works quite well. The most difficult thing you're dealing with is that the thing is just *too* high/far away; you're getting pretty decent results for the conditions you're shooting under.
I usually shoot the 100-400 in manual at f/5.6 or f/8 and between 1/500 and 1/1600 for birds flying; adjusting ISO for exposure adjustments.
You, really, just need to get closer; either by finding a location where they'll fly a lot lower, or throwing a 1.4x TC on there. Although, I'm not certain the 1.4 will allow AF in that combination on a 6D...I know they enabled f/8 AF in the 5DIII firmware a little while back.
From there, work on your stance and holding.
- Feet shoulder width apart, elbows tucked in at your sides for stability (if you're chicken-winging it while holding the camera up you're going to get a fair amount of jitter).
- Acquire your subject a fair distance out and begin AF; tracking by turning your entire upper torso.
- Press and hold the shutter, smoothly, don't "jab" it
- Follow through on your pan for a second after you release the shutter button
Some examples from the 100-400, when the birds were much closer. Minimal cropping needed with these; click through to Flickr for better resolution
RTH: 400mm on 7D, f/5.6, 1/1600, ISO 640

IMAGE LINK: https://flic.kr/p/iVZtMj
_MG_0064
by
Guideon72
, on Flickr
Osprey: 400mm on 7D, f/5.6, 1/640, ISO 400
IMAGE LINK: https://flic.kr/p/f8XrCx
Osprey hover-6571
by
Guideon72
, on Flickr
Tundra Swan: 400mm on 7D, f/5.6, 1/640, ISO 200
IMAGE LINK: https://flic.kr/p/iZUQXn
stray Tundra Swan-0224
by
Guideon72
, on Flickr
Brown Pelican: 400mm on 7D, f/8, 1/640, ISO 400
IMAGE LINK: https://flic.kr/p/nWSMiS
Chinook Brown Pelican-3533
by
Guideon72
, on Flickr