First, humans see horizontally, so a horizontally-oriented picture is what we usually prefer.
Second, 35mm film evolved from movie films at the time, and those happened to have a 24x36mm recording surface. Most digital SLRs evolved from 35mm SLRs, so they kept the 3:2 ratio, Olympus being the exception since they like 4:3 for some reason.
Medium format films come in all sorts of sizes, both rectangular and square, since it was intended for photographic purposes from the start.




