I thought of doing that when I bought my 500/4, but am very glad I didn't. There are those times I just don't want to lug the monster around and the 400 is so easily handheld.
I'm thinking that way about my new Sigma 120-300. It's a great lens, but man it's hefty. The 400 feels like air after using the Siggy.