Since you don't use that range very much, it doesn't make a lot of sense to go out and spend that type of money on it. I will say, though, that since you don't seem to be that happy with the 100-300 you have, it still wouldn't hurt to upgrade it, but just not spend so much money. Perhaps something like a 55-250 IS would do the job and only cost you around $250 for a new one.
That 17-50 you have is a pretty nice lens, so if you're happy with it, there's really no need to upgrade it at this point. You mentioned the 24-105, so I'm left to wonder if you simply want more focal length; if that's the case and you have the cash for it, the 24-105 is an excellent walkaround/general purpose lens. Even on a cropped sensor, the FL range is very useful, as long as you don't need anything extra long or extra wide.
I took a look at the lenses in your sig and it looks like you've got a good bit of overlapping in focal lengths, so depending on what you decide to do in the way of upgrades, you might want to consider getting rid of the lesser quality lenses as you upgrade to better lenses. Here are some ideas:
--swap out 100-300 for 55-250 IS
--If you upgrade to a 24-105 (or 28-135), definitely get rid of the 28-80. To keep something wider handy, decide between either the 18-55 IS or 17-50. Later on, you might decide you want something wider, but you'll at least cover moderately wide until then.
HTH,
Heather 