I might be the wrong person to answer, because I'm a big fan of IS on longer lenses. In fact, it's one of the main reasons I switched to Canon back in 2001, and three of the four lenses I bought initially had IS. At that time, Canon was the only game in town... the only manufacturer offering any form of stabilization. It's gotten me shots I wouldn't have been able to get without it and I'm convinced it's improved other shots as well. Len-based IS such as Canon uses helps steady the viewfinder image, too, which in turn can help when tracking and shooting moving subjects. Today I have seven IS lenses and still swear by it on longer focal lengths (I'll take it when available, but it's not a really high priority for me on shorter focal lengths or on macro lenses). When I switched from film to digital and started using my lenses on crop sensor DSLRs a lot, which amplify camera shake issues by 1.6X or 1.3X, right along with the effective focal length, IS on longer lenses became even more important to me.
So I'd get the 70-200/4 IS USM, personally. In fact I have the f4 IS, use it a lot and really like it. It's sharp enough to make your eyes bleed, as they say. I also have the 70-200/2.8 IS "Mark I", which is still a very, very good lens too. (You also might consider a used one of those, just watch out for hard-used ones like mine, it's been a popular lens among pros who typically work it pretty hard).
Other considerations are that the 70-200/2.8 non-IS is the oldest 70-200 design in the Canon line-up. Now, all of the Canon 70-200s are top quality build with great image quality, but many who have compared them more closely than me feel the IQ of the 70-200/2.8 non-IS is the weakest of the bunch. By a small margin, I'm sure. The f4 non-IS and original f2.8 IS are fairly neck in neck. Meanwhile, the 70-200/4 IS is the second-newest model in the line-up and holds it's own in image quality even against the latest and greatest 70-200/2.8 IS Mark II.
The f4 lenses are about 1/3 lighter and smaller than any of the f2.8 lenses, too. The f4 lenses use a 62mm filter, while the f2.8 lenses use 77mm, for example. Since they all are L-series, they all come with a matching lens hood (the f2.8 lenses have a "tulip" shape hood, while the f4 lenses' hood is standard design). On a long day's shoot, I'd much rather handhold the f4 lens, than any of the f2.8.
Of course, the f2.8 lenses can render a bit stronger background blur effect. But since you have a fast prime in your 85/1.8 (which can render even more background blur than the 70-200/2.8), this might be a moot point and you may be happy with the smaller/lighter lower cost 70-200/4 IS.
One reason the f2.8 lenses sell for the prices they do is that they all come with a tripod mounting ring. That's sold separately for the f4 lenses. The optional Canon tripod mounting ring for the f4s is rather pricey, but there are considerably lower cost third party "clones" of it available, which many people use instead and seem happy with.
The original f2.8 IS was rated for 2 to 3 stops worth of stabilization. The f2.8 IS Mark II and the f4 IS both have newer form of IS that's rated for 3 to 4 stops worth of stabilization. All three have "self-detecting" IS, that turns itself off if the lens is solidly locked down on a tripod (unlike some "lesser" IS lenses such as the 300/4L IS, 28-135 IS and 24-105L IS... where IS should be manually turned off if locked down on a tripod to prevent the IS from going into sort of a feedback loop where in absence of any movement it actually creates movement, rather than correcting for it.)
Both f2.8 and f4 lenses are very fast focusing in good light. Because the f2.8 lenses allow in more light for the camera's AF to work with, AF performance in challenging situations is a wee bit better with the f2.8 lenses. It's hardly noticeable though, to be fair. OTOH, most Canon cameras have at least the center AF point optimized for improved low light performance with f2.8 and faster lenses.
Dunno if this helps... Have fun shopping!
That's when I realized Image stabilization, especially on long lens is really important. 
