As already said, regardless of arrogance, only one view is correct in the United States - that of the street photographers. This has been discussed here and elsewhere time and time again.
It really isn't important what the subject is. More importantly, there isn't a reason to suspect that a street photographer has any nefarious intentions unless something else is going on.
I am a parent of two young children. I understand the urge to protect them at all costs. However, there is a problem when parents perceive danger where there is no danger. Are you also saying that I should be breaking high resolution security cameras, because I don't know what will be done with the footage taken by them?
As a sometimes street photographer, I have had these confrontations. I have had them with police, parents, and boyfriends. I am always happy to have a civil discussion with anyone regarding something I consider a hobby and an art. Most of the time, I will even willingly delete pictures. However, I am not willing to have my rights or safety violated. In this case I would have probably called 911 and readied my pepper spray the instant someone said I wasn't going to leave.