IMO, 2.8 zooms zap the fun out of photography. Some folks swear by them, and I swear those folks have sore shoulders
I own a 24-105 F/4 IS and have used the 24-70 f/2.8 II. I honestly don't find the 24-70 much heavier. I have a 5DIII and very often shoot with the 580EXII flash onboard, either for indoor bounce or outdoor fill. In such a configuration, the overall package with the 24-70 is less than 7% heavier compared to the 24-105. At that focal range, f/2.8 makes a difference in subject isolation and the 24-70 has better low-light autofocus.
I will agree with you about the weight of the 2.8 vs f/4 versions of the 70-200. That's like an extra 2 pounds for the 2.8. For what I shoot, f/4 is fine and I don't have to hesitate whether or not to bring it.