A number of street photographers have done just fine without a longer lens and without annoying pedestrians. Longer lenses allow you to distance yourself from the subject, and they're great for street portraits, but they also increase the chance of obstructions, like other people or vehicles. Moreover, the distance one might need to step back might not be available, as busy streets or buildings tend to get in the way in an urban environment...I've encountered this problem even with a 50mm on a film camera (full frame).
Plus, if you are trying to avoid looking like a creep, your subject might not notice you, but someone passing by you and your long lens will likely think the worse. Then you have to ask yourself, does that really matter? F' the judgmental bastards, right?
Also, I've found that standing on a corner for awhile, observing patterns and such, can be beneficial. Most everyone else is in transit, and thus they don't know how long you've been present, and more importantly, don't care. If you have a small camera, they often won't even know you're holding one.
Again, what is the objective; to record life and its surroundings or to make friends? I've already made it clear that I have my personal limits, Gilden has his. In Japan, I caught a couple of youths taking snaps of me, usually with their cameras at chest height. I only noticed, because I was checking out their cameras, something the average human probably wouldn't notice. Anyway, I didn't care…security cameras catch me a lot more.