Unless you really like macro I would not use the 100L for portraits in your case. I also have the 24-105 and the 100 macro non L, for portraits besides the sharpness of the macro there is little difference between the 2 in the separation and bokeh department. You will see a much more noticable difference when you compare your f4 lens to the f2 of the 135L or to the f1.4 of the sigma. I was researching a possible upgrade to the macro L for the IS and the weather sealing, but once I saw the bokeh of the 135L I fell in love and I got it instead of upgrading my macro. If you want I can do a quick comparison of the 3 lenses wide open.