I see the 35L is very popular, BUT... I believe the view from a 50mm perspective is much more pleasing to the eye. I have the 50L on my 5dII FF and hardly ever take it off. If I need a wider angle, I reach for my 24L. I have owned the 35L and enjoyed it, but sold it because I hardly ever used it. I found the perspective a little "boring", and people at the edges look a little distorted. I found it was not wide enough for architecture and landscapes, and not close enough for informal portraits. The 50L is also much better in the dark, which comes in handy ALOT, because having the big 580EX speedlite at a informal social gathering is too attention grabbing. I also started out with the 50mm f/1.4, which is a wonderful lens, but compared to the L 1.2, it looks a little "flat", contrast is worse, and has much more purple fringing. On a budget, it should do you fine.
For tighter shots, I have owned the 85L, and find the 135L much more useful. I believe the 135L can produce the same quality portraits (if not better) as the 85L and I can use it for action sports as well.
To sum it up, after owning the 24L, 35L, 50L, 85L and the 135L (not to mention the 200L f/1.8 and the 300L 2.8, but those are hardly walk-around), I love the images from my 135L the best, the 50L is my jack of all trades, and the 24L is when I need to go wide.
Just to mention, this is my first post on this site. I have been lurking for a while. I have been shooting SLR's since the late 70's. I look forward to contributing my advice on this very helpful site.