I owned both the 300 2.8, 500 4, but recently sold the 300
. For birding, the 300 with tcs just wasn't long enough. For other wildlife, it just doesn't have the versatility. And to carry a 500 around, it's basically impossible to get too far with a 300 hanging around your neck which is why I prefer the 100-400. Although the 300mm performance is stellar for the shots it gets, it's range is a bit too limiting imo especially if you already own the 500. The 100-400 is a relatively light lens, and it wouldn't be too big a deal to tote that around along with the 500.
And as far as the 300 f4, again a nice lens, but I prefer the versatility of the 100-400. I found the 100-400 comparable if not better in IQ to the 300 f4 with 1.4x TC.