Like a few things in photography, the bag is a compromise.
As quite a few of us have said on more than one occasion, one bag will not do everything. Taking everything with you 'just in case' is a nice thought, but it soon gets wearying. Quite often, when setting out, you will know, roughly, what you are expecting to see - so you pack your bag accordingly.
For example, if it is a night shoot, I know I will want my remote. I will have the lens I want to use in mind, the tripod will be strapped on my back - and, having thought it out, off I go. On site I may think "Oh, I wish I had bought xyz with me." Well, tough titty, I didn't - so I improvise. That's what we learn to do.
Lenses are the same - I am convinced that there isn't one lens for all occasions. Sure, the Sigma 50-500 looks like a brilliant range - but it's size hardly makes it a walk-around - it stands out in a crowd and it's aperture speeds means it really prefers to be used in bright light. A bit like the 100-400. Go have a read in the Lens forum about the reasons for choosing a 400mm prime over the 100-400 - there are as many opinions as there are posters. Each and every one of them thinks they are right! 
Compromise.
That's what we all do when looking at what's available and what we think we need.
Hell, sometimes it is budget that is the primary factor. I know it was when I bought the Sigma 24-70 rather than the Canon version.
We learn to work with and within our choices - and having to think our way through or around that, I think, makes us better, more versatile photographers.
Sure, the advice of people who have been there, done that it valuable - but that is all it is - advice based on their experience. You still have to make the final evaluation.
Sorry, it seems like a rant. It is not meant to be - just another way of looking at things.