I did a quick bokeh ball test between the 70-200mm f/4L and the f/2.8L IS mk2 last weekend. Pictures were taken with the camera body mounted to the tripod, not the lens. You mentioned the f/4L but seem to cringe at not having IS, but with the 5D2 you have plenty of ISO performance to allow hand holding. Also, you'll most likely be walking around using the lens you buy in the daylight, right? Plenty of light available to matter whether you have IS or not.
5D mk2 with 70-200mm f/4L and 70-200mm f/2.8L IS mk2 (IS turned off)
70mm, MFD @ 1.2 meters, manual settings for same exposure, I don't remember what it was set to exactly. This was purely to test the quality of the bokeh ball, nothing else.
Which is which?