I'm the other way around. I'm more of a tele guy. I've not played around that much with a focal length of 35mm (full-frame terms). However the times I did, I found it very difficult to get interesting shots. With a tele lens it's often a piece of cake since you can easily isolate your subject thanks to the narrow depth of field. Much less distractions noticeable.
Different strokes for different folks =)
I prefer wider FOV for portraits specifically because they don't completely nuke the background to an abstract blur of color, allowing to give your portraits context and to tell a story with a shot. Also Wide focal lengths + fast apertures give a very interesting three dimensional subject isolation that you can't achieve when the background elements are taken completely out of the frame.
[IMAGE'S LINK: https://flic.kr/p/qdbZKz]DSC06963 by Carlo Alcala, on Flickr
[IMAGE'S LINK: https://flic.kr/p/mf412h]DZ2A8786 by Carlo Alcala, on Flickr