One thing that I can think of is: if you only use your zoom to frame your subject from where you are, then you are choosing a focal length based on convenience, not based on your vision for the photograph. In my opinion, that's a sure-fire way to end up with pretty mediocre photographs a lot of the time. When I first started, I did that as well. Now what I do is more like this: for large objects, I move to where the play between the subject and other leading elements combine for the best composition, then choose the focal length that will isolate that composition (perspective has everything to do with distance, not focal length, so I find the perspective I want and choose the FL best suited to the shot). This is similar to what you're saying, but that's just choosing the correct focal length for the shot...what zooms are for. OR, for smaller subjects, or for wide-angle shots, I find the foreground subject and background I want, and I look at combinations of focal length and movement changes so that I get the most pleasing view....(for instance, if I have flowers at the front, with mountains behind, too wide makes the mountains too small to have impact, too long makes the flowers not jump out as the leading subject).
Of course, there are instances where you can't move, and you want to frame a shot a certain way, so you zoom to frame. Or there are areas where the focal length doesn't matter because there's just one subject you want to isolate (such as with sports, isolated people or wildlife portraits (with no play in the background). However, any situation where you've got play between foreground and background requires much more thought to the focal length and perspective you want to use.
Essentially, use a focal length because it allows you to get the framing / perspective / balance between elements, not because you don't want to move.
All that is now out of the way: To zoom vs. prime, it's really about what you're shooting, and the time you have. If you need to switch focal lengths often while on a shoot, a zoom is obviously the best choice. If you need fast lenses, primes are often the only way to do it. Also, optical quality tends to be higher with primes. If I could afford a good compliment of zooms / primes in each focal length, I would, but I'm not there yet. The 35 f/1.4L is calling to me (though I will probably get the 35 f/2 until I can stash away some cash). I use my 50 f/1.8 all the time, yes, mostly for portraits, but not only for studio type setups...it's very easy to frame the subject by walking a bit.