Heh, even old ladies will bark at you if they spot you with a camera pointed in their direction.
I rarely experience confrontations, but this very thing happened to me a couple of weeks ago while standing in the aisle on a Manhattan bus. I wasn't taking any photos, but i needed to look through the viewfinder to adjust some settings in the menu. I didn't want to point the camera down toward the woman seated beside me, so I chose to point it down the aisle with most of the passengers' backs toward me. In public places I'll sometimes cover the lens with my free hand so as to diffuse any alarmists, but to my regret I didn't do it this time. Naturally, there was an old crow of a woman berating me shortly thereafter. I tried to explain what I was doing and that I didn't have the slightest interest in her photo, but that didn't satisfy her. I went further, stating that even if I wanted to take her image I was in my right. Then she made some idle threat about taking the camera from me. So I said, "in that case", while shooting several images of her, "I am going to take a few," explaining how I'd use them to ID her if she so much as touched me or my equipment.