I'm with gasrocks - pick the lens to do the job. If I am am walking around a zoom is my preferred lens because it allows me to frame the shot I need.
But really - it all boils down to personal preference. If you don't mind changing lenses - go for it. I personally think it's a pain in the butt. Searching for the lens in hte bag, juggling the two lenses before putting one back on hte body, doing it quickly to avoid dust, worrying about dropping my babies. If I can avoid it, I will. I'm not afraid of it, but it's an oppportunity for problems.