Well I may be talking sideways here since I'm a woman, buuut if some guy in public started taking pictures of my kids without permission, I'd probably get very uneasy and confront him - pro or not (I chuckled at the Dexter comment). And I love candids! It's all in the context. At a school trip, sporting event, etc, one would expect to have some level of photo/video documentation; for our school functions etc we often have to sign a photography/media consent form. That, I have no issue with. I also have no issue with going up to someone and asking if it's ok I include their children in my photos.
We also have to be a bit culturally sensitive, I think. I've travelled to countries where it is illegal for a man to take photographs of a woman that is not his wife (ie just about anywhere in public) so you do NOT see people walking around with cameras hanging off them, taking candid shots. (even though I love candid travel photos) I live in the West and we do have those freedoms, but remember that some people don't, or didn't, and it may be new to them or uncomfortable for them, even to the point of feeling violated. So that said, it's always wise to ask permission, or to be very, very discreet (ie unnoticed).
I shot at my kids' racing meet this morning (at the request of the club) and parents signed a consent form when they signed up, and I still get asked by some folks, "sooooo... are you taking shots of ALL the kids, or just yours, or..." I am very new to being "the one with the big white lens" but it garners attention, and FAST. So I am having to think on my feet and just flat out ask if that's ok or not ok with them, I can quite easily work around any special requests not to be included in the shots. Or like Sue... if the opportunity is there, I'll pre-empt it, with a, "hey, do you mind if..." scenario.
JMHO, of course.