I don't think it's crazy. If the 7D has more features than you'll ever need, then why pay the premium for them to never use them? Since the consensus seems to be to spend more money on glass than the body, scaling back from the 7D to get better glass fits that axiom perfectly.
I'm in the midst of deciding which body to buy next (upgrading my Rebel XT). There are things about the 7D I like. But there are things I do not. Video is not something high on my list. And the larger file sizes that RAW files will be from the 7D mean that I'll run out of disk drive space faster than a camera with less megapixels.
So for me I'm thinking 1). 30D (mostly due to lower cost), 2). 40D due to MP still not too high and [to some people] better low-light handling than the 50D, or 3). 50D due to better display and the micro-adjustment feature for lenses.