That depends, Roy, on the local regulations, and where in the city hall or state capitol you are taking pictures. The bathrooms, for example, are usually off-limits.
A public place is often confused with a place that has public access. Many places, such as malls and store, have free public access. That does not make them public places, just places open to the public.
Also, places owned by the public may or may not be public places. Schools are typically owned by the public, but they are not public places. They are regulated public properties, which would perhaps better be thought of as government-owned places. The government, in this case, has the status of a private entity. The fact that you are "part owner" of the government is irrelevant.
Parks, beaches, and, yes, Area 51 fall into the govenment-ownership category. Access is defined by regulation. In the case of a beach, the regulations are simple (and usually obvious), and may include hours of operation, no alcohol, no dogs, etc. In the case of Area 51, the regulations are a bit more strict.
Another way of looking at it. Let's say you own a large block of Microsoft stock. This makes you a part owner of Microsoft. Does this give you leave to freely go where you want and photgraph what you wish on Microsoft's property? Not on your life! Part ownership of an entity does not grant license.
I was reading recently about a case of a conflict of laws. A man was asked to leave a store because he was black. This was made clear in the way he was asked to leave. This is clearly against the law in the U.S.
The man refused to leave, and was arrested for trespassing. As the courts ruled (and I agree), when asked to leave by the owners, his failure to leave constituted trespass. He was fined accordingly. Why he was asked to leave was a different issue entirely, and was irrelevant to the owners right to deny access to his property.
The man did (and properly so) file a discrimination lawsuit against the store owner. He won the lawsuit. The owner had no right (under U.S. law) to ask someone to leave solely because of their race. This is an entirely different issue from property rights.
What it boils down to is, if asked to leave private property by the owner or his representative, you must do so under the law, regardless of why they asked you to leave. If you think they were wrong in asking you to leave, file a lawsuit.
Remember that "Sue" is not only the name of a pretty girl, it is also a verb.