If you want a lens specifically for portraits, I'd say the 85 1.8 is a better choice. If you want a lens that can do portraits and a lot of other stuff as well, the 70-200 is pretty much perfect. You say you take portraits, sports, landscapes etc... So I think the 70-200 would be a better choice for you.
As for the f4 IS vs f4 non IS, I have owned both. I now own the non IS. The reason for this is that I don't feel that the addition of IS (and weather sealing because at the moment my camera does not have that feature), is worth ~$600. In terms of IQ, the IS version is slightly sharper at the corners, but in centre frame, the two lenses are equal. There is no big difference, as some people make out.
The $600 saved can also go towards buying a very good tripod which will help far more than IS for many photos. IS is only really useful when you are shooting static objects in low light. If you don't see this as being a major part of what you shoot, you don't really need IS.
If you do decide you want IS, then I'd recommend the sigma 70-200 2.8 OS over the canon f4 IS, because it is the same price, but offers you better build quality and an extra stop of aperture. Or I'd recommend waiting for the tamron 70-200 2.8 VC to be released since their current 70-200 2.8 is amazing in terms of IQ, but lacks stabilisation and a good AF motor.
As you can probably tell, I was distinctly underwhelmed by the 70-200 f4 IS in terms of price/performance.
The only case in which I would recommend that lens (or buy it myself) would be if you need a light, compact 70-200 with weather sealing, because all the others which have weather sealing are 2.8, and are large and heavy. In all other situations, I recommend either the f4 non IS or one of the 2.8's.