to answer your questions in order:
I don't use the 24-105 very much if ever. it's definitely going to go. I use the 17-40 but am always frustrated with f/4 maximum. I don't often use the 70-200 or the 100-400, but I do like to dabble with nature shots, just haven't had time since the baby was born.
I want wider apertures. I always find myself turning up ISO and being frustrated with grain or sacrificing shutter speed and getting motion blur. I'd also like to be able to get more background blur than I currently can. Also, I don't shoot any of my zooms wide open, because I am unhappy with the results. I shoot the 70-200 at f/4, and the 100-400 at f/8.
I wish I had a lot of lenses. don't you? I would love to have an 85 f/1.2, but can't justify the expense, so I am rearranging the prime lineup to have minimal overlap, starting with the 50mm which I already have.
I have a new baby, which the 50 is great for. it's almost all that is on my camera. I'd definitely like a wider and longer option for different circumstances though. that's why I was thinking 24 and 135. I like the interval of focal length.
lastly, I think the 50L will go to f/1.2 instead of just f/1.4. Also, it should be sharper at all identical f/stop settings. The only reason I am considering trading out for the L is because according to my math, it is feasible without investing any more money.
So, do you have any thoughts on the matter? I know it is my choice and everything. I've already thought through my own issues, that's why I am asking what other people think. Not looking for counseling here, more just opinions on the matter.