Are you trying to shoot portraits at f/1.2? 1.2-2.8 are great apertures for creative portraits, or isolating an individual in candids, or artistic shots of any type, but there's nothing wrong with an f/2.8-f/5.6 portrait with the 50L. I love shooting shallow (1.2-2.0) for distance, environmental, portraits to get pop, but there are issues in doing so. High contrast will give you CA(any fast prime), which will make the images look soft. You need to learn what lighting works for these shots, and what doesn't. Shooting shallow, closeup gets old after a while, but always makes for a nice image or two as well. Shooting headshot/bust shots with the 50L, at f/2.8-f/5.6 will give you a VERY sharp image, with excellent background bokeh. No regrets here.
The 50L makes a great portrait lens for me, on a crop or full frame. But, everyone is different. A friend of mine prefers 85mm on her crop-body, which I tend to leave for my FF or 1d. She'd never consider 35mm for portrait work, where I'd use it on a crop, but not as much on FF. Its all in your style. Nothing wrong with the 50L, if you like the focal length.