Seems the argument that the owner could invalidate the rights to use if he sold it is a little odd.
At the time of the permission, the image was shot. So, later, the new owner has no standing, as the time frame of shooting it was then, not now. I can't imagine a retroactive revocation of an image right holding water. If so, images of any 'famous' person could be withdrawn and cause harm to the original purchaser after that purchaser spent time and money, for example.
Let's say, Nike shot an image of a BB player in Nike shoes - because he is famous. That athlete can not withdraw the rights to show him wearing Nike shoes, because Nike shoes is the shot, and incidental is the famous athlete - an implied endorsement.
Although the house in question here is not an endorsement per se, it still divorces itself from its owner. It is a time and place in history, and not a removable icon.