I was in a similar dilemma a while ago and picked the 24-105. I'll try to list some reasons / thoughts as to why I got to that conclusion:
- I already owned the 28 1.8, 50 1.8 and 85 1.8 which work great for low light and shallow DoF, but I wanted a zoom for easy access to different focal lengths and faster shooting.
- Since I had no plans of ditching my primes (love them), the added range of the 24-105 was nice.
- IS really helps me when I'm shooting at smaller apertures (in really low light I normally get the 28 or 85mm primes)
- The 24-105 was much cheaper as there's a ton of nearly new used ones on the market. I bought mine for 625 euros which is easily 300 euros below new price, but it was as good as new and only used twice.
- The 24-105 is lighter and more compact, fitting better with my style of shooting (location + quite a bit of strobist).
If forced (for example - financially), I could probably consolidate the 28 1.8, 50 1.8, 85 1.8 and 24-105 into one 24-70, but I'd miss the short MFD of the 28mm, the great low light and portrait capabilities of the 85 1.8 as well as the IS on the 24-105. Another thing that's quite important to me is being able to travel compact and light, so with just the 24-105 when I want, or maybe with the 28 and 85 1.8 if I plan to encounter low light situations.
All of my lenses have their place in the line up and for me, the 24-105 fit best.
In short, I feel that the 24-105 couples really nicely with a fast 50 or 85mm prime, if I didn't have any primes, I may have chosen the 24-70 for the extra stop and better bokeh. In my situation though, the 85 1.8 takes on that role, when the situation comes up.