I bring my 40D on the hill all the time. I've been riding for about 12 years now and have a pretty good comfort level carrying a bag with my camera gear. I have had some terribly experiences though. Last winter I was on a trip to Kirkwood CA on a powder day and took my camera out to snap a few shots and a gust of wind blew a bunch of snow into my bag! it was pretty much filled. I didn't have much in it and nothing was damaged but i still had to ride down some pretty treacherous terrain with my 40D in hand cause i had no time to clean out my bag on the hill. Also, this past weekend, getting off the chair lift @ mount snow in Vermont a buddy of mine got his board stuck on mine and we both tumbled, me with camera in hand. I was able to save the camera and I took the brunt of the fall. Just remember to be cautions and always put the camera first in terms of awareness and you will be fine.
Here's a snaps from this weekend I took with my 40D

