I just got mine after debating between it and the 70-200 f4 IS.
For me, the 70-200 f4 IS doesn't really make sense since at 200mm, the 70-300 is only at f5 or so. I'm ok with losing f4 at 200mm and gaining 300mm in a more compact, albeit heavier, package.
Also, the IS on the 70-300 is way quieter and smooth. I hardly sense it at all, as opposed to the 70-200 which constantly makes whirring noises when IS is engaged.
As far as price, the 70-300 is just $150 more. It was a no brainer for me. 70-300 is far more compact, as sharp or sharper, better IS, and 300. The only negative compared to the 70-200 is 200mm at f5 and a bit heavier weight.
The 100-400 is nice but its size puts it out of the realm of portability for me. The 70-300L is touching the limits of what I can use somewhat discreetly in public (as discreetly as a white lens can be anyway).