Hi msteudel. I was in the same boat about a year ago. I tried to decide between the 400D and the 30D. I ended up paying a whole of a lot more money on the 30D and I personally hate it. I sincerely regret buying it. And yeah, the relevant point in this heated reply is that I went for the same consideration: I really liked how the 30D feels in my hand.
But take it from me, this is not a very intelligent decision factor. I mean yeah, for a pro this could be the most serious factor to consider cuz' you just do your job and get paid for it. But when I go hiking during summer times and have a brick around my neck...Nope, it's not worth it. I mean if you're an amateur why consider the "feel"? Nobody will ever force you to shoot for hours!
So I'll say it again: big cameras are worthless for amateurs. I strongly recommend against the 40D because I was in the same situation as you are. Oh, and the 500D has better ISO performance and for me, that's the most important thing.
My last advice for you is this: forget DSLR's and buy something like the G11 or the S90. I mean, just look into it, it won't hurt. From what I'm seeing, these cameras will deliver results comparable with my 30D at the same ISO setting. But they also have fast lenses, they're small and shoot video. I'd think about it if I were you. If you don't have fast lenses, you'll soon come to a point where you must use your flash and a guy with some compact shoots without it. I was there. It's simply humiliating. So really, properly analyze your true requirements and look into good compacts. You might get a surprise.
Later edit: actually there's one more BIG advantage of compacts (in my view): they have the damn AF light! Why is it so hard for Canon to put a LED or something on their DSLR's? The reason I'm so obsessed over this AF-light is because Canon DSLR's with no external flashes are 99% impossible to use at night-clubs and other parties with very low light. First of all, everybody hates you for the 5 seconds of flashing needed for proper AF and when you finally get focus lock everybody's gone because they thought you already took the picture (because you were bursting the flash strobe for several seconds). So you end up with crappy out of focus shots of nervous people. And a guy with a compact comes along, the camera uses the AF-light and everybody's waiting for the flash. Because they know that the picture is taken after the "orange light". Again, that's very sad.
OK, I'm putting on my flame-suit...