I avoided buying the 400 lens for wildlife as it is just too heavy. I have the 300 2.8 and the 500 and 600 f/4's. They all have their place in one repertoire depending on what you are shooting.
I have used the 300 with a 2.0TC on it and I didn't like the loss of quality and light that you end up with. It is great with a 1.4, perhaps it is just my copy of the 2.0. It was ok for animals/birds that weren't in motion but once you get a bird in flight I just wasn't happy.
The 500 and 600 are great birding lenses and you can sometimes be too close to larger animals so that is where the 300 comes into play more often. I went to the 600 after getting my full frame bodies as there were times when I didn't have the reach that I wanted when using these with the 500. Both the 500 and 600 have excellent sharpness to them but certainly not to the extent of the 300. The 300 is one of Canon's sharpest long lenses, probably only second to the 400.
The 300 is very much handholdable, even with the 1.4TC. The 500 can be and I have but you don't do it for very long at a time. The 600 is almost impossible to handhold unless you have a shutter speed that is way up there and it will tire you out pretty quick. With all of these lenses you have to consider a good support system. When I had just the 300 I had a Gitzo carbon fiber tripod with a Kirk BH1 head with a Wimberly sidekick. This did work with the 500 but there were times where it didn't feel comfortable. You really had to make sure that your knobs were turned in tight. When I got the 600 I knew there was no way that this was going to work so I invested in another tripod and the Wimberly Head Version II. It is a sweet unit. If one was going to get the 400 you definitely would need something in the line of the Wimberly head given its weight.
A smart photographer will fully investigate these lenses before making a purchase of this stature. It is not about not deserving them, its about making an informed decision. You can't make that decision unless you get the information.