If you are on public property, you have a right to stand there and shoot. If the object (building) is in the photo, as long as you are not using the photo for commercial purposes (to advertise your business, or to sell for someone else to use to promote their business, nor to simply sell the image to someone else) you do not have to have a property release. If the object (building) is in the photo, and you are using the photo for commercial purposes (to advertise your business, or to sell for someone else to use to promote their business, nor to simply sell the image to someone else) you do have to have a property release before it can put put to use for any commercial purposes.
Now, in the age of terrorism, it is common for public buildings to be 'controlled' locations with some restrictions. You could be assisting a terrorist who would be bombing the building or you could be plotting to poison the water supply, for example.