It's been hinted upon, but in addition to the camera/lens not keeping up with the focus, insufficient shutter speed could be a culprit as well. Depending on what the dog is doing, you may need a very fast shutter speed. For a dog just trotting along, 1/500 or even a little slower should be plenty. If the dog is really moving, you may need something more like 1/1000 to freeze its movement. I own a 5DMII as well, and even though it definitely doesn't have the best autofocus system out there (it really wasn't designed for action) and I tend to switch to my 40D for action shots, I've found that it still seems to do a pretty good job tracking a subject once it's locked on to its focus point. It's generally just in that initial focus where I find it can be slow. It does require a steady hand, though, because once you move off that focus point, it can often be a bit slow to reacquire.
Oh, and good question by Nature Nut. The 5DMII (and most of the other non 1-series cameras) use a simpler algorithm (that's not as accurate) for the outside focus areas. It's usually not a big problem with static subjects, but it can definitely be a bit inconsistent with moving objects.