Okay I didn't know if I was weird for doing this. I feel like the less I take it off, the less likely it is to get dust inside the lens or on the mirror.
But I only have one decent L lens (24-105) that I constantly use as a walk around. If using another, it is because I rented a 70-200 or something.
The two pros I know here always take the lens off, but I think that has more to do with the fact that they have a huge selection of lenses and tend to put the body only in the Pelican carrying case for assignments.