I should have done more comparing before I sold my 5D2, but as soon as I experienced the AF on the new 5D3, I put my Mk II on the market. From what I'm gathering on the internet, they're similar at lower ISO settings (though the Mk III has considerably less banding if you try to pull up shadows a great deal), but at 3200, the Mk III has a noticeable advantage in noise performance, and at 6400 and 12,800, the difference grows larger. I know that on my Mk II, ISO 12,800 was a bit of a "crap-shoot" with shadow banding noise messing up quite a few pictures. I realize, of course, that 12,800 is a bit extreme, but if you need it, you use it.
I would say that for low-light shooting at 3200 and up, the Mk III is considerably better, both in terms of noise and how well it will focus.