I have all three: 50/1.4, 85/1.8 and 100/2.8 USM (and a few more)...
I really haven't formally compared the 100 and 85. I simply use the 85 for portraits and the 100 for macro. Unless I happen to only have one or the other with me, and need to grab a portrait with the 100, or put an extension tube behind the 85mm to use it for a macro/close-up shot.
On a crop camera, 100mm can be a little long for portraiture. Not a problem if you have plenty of working room.
Also, like any macro lens, the 100/2.8 is a bit slower focusing (though the USM and focus limiter help a lot). The 85/1.8 is pretty instantaneous focus, nice for fleeting, candid moments. (The 135/2 is also very fast focusing, but a little long on crop... I use it more for portraiture on FF.)
The extra 1.3 stops of f1.8 can be useful shooting portraits.
The 85mm is also smaller and less obtrusive.
The 100mm is sharper.... but sharpness isn't always a good thing for portraits.
If only occasionally shooting portraits, mostly interested in macro.... Keep the 100/2.8. Especially since you have 50/1.4 as an alternative portrait lens, when you need a bigger aperture, faster focus or don't have as much working distance.