My first DSLR was a Rebel XT (350d), and after using my cousin's 40D for about 5 minutes, instantly knew that I needed to upgrade. The 40D was an upgrade in every way from the 350d, the speed, image quality, lcd, larger viewfinder, weight/ergonomics, all were a huge upgrade.
In the last year, I have a purchased, in order, a 5d, 7d, and 60d. I truly believe that every single one of these cameras is an upgrade from the 40d, albeit in different ways. For straight IQ, I still prefer the 5d. Sure it is old, clunky, and slow, but it puts out some amazing images. The rear LCD is really bad on the 5d also, but once you get the pictures on the computer, who cares?? I really believe I will eventually go to all full-frame, and wanted the 6d, but knowing I actually needed to buy two more bodies, I just wasn't financially there yet, as I am not set-up glass wise to transition to all full-frame. If you want to try out a FF body on a budget, I would without a doubt recommend the 5d, you can find them used for around $500.
The 7D and 60D are also both big upgrades from the 40d. The image quality isn't all that different than the 40d, but they are both definitely better at higher ISOs, and I really don't mind shooting even ISO 12800 in a pinch on either of them as they can both be cleaned up fairly nicely. I would much rather have a sharp, noisy image than a blurry one.
The overall speed and AF quality of the 7d are a huge upgrade from the 40d. I find the 60d focuses faster than the 40d did also. The 7d and 60d both have great rear LCD's, with the 60d having the advantage of the swivel screen if you need it. I much much much prefer the layour and ergonomics of the 7d over the 60d, but it's something that you could get used to over time. More than anything, the 7d is just more similar to the 40d and 5d, whereas the 60d is totally different. The video on the 60d and 7d are also quite fun to play with.
If you are on a budget, I would recommend the 5d or 60d as upgrades (both around $500). If money is not as much of an issue, pick up the 6D if you want FF, or the 7D if you want to stick with crop.