Well, only you can answer whether it's worth it or not...
I'll tell you... I faced the same thoughts last year when I finally splurged and got the 24-105L. Alot of $$, no doubt. I still don't know whether I'm convinced it's worth it or not. I'm Joe-Ameteur who happens to really have a strong interest in photography. Most of my shooting is my two little girls... surely don't have any aspirations ever to make any $ as a photographer.
That said, I think it really depends on 1) your financial condition, 2) the level of interest you have and how enduring that interest will be.
Is the image quality better? I went from a 28-135 to the 24-105 (similar move your're contemplating) and the answer is yes. Color and saturation seems richer as well. Is it worth the extra $$.... tough call. In my case, I think this is my walk-around lens I'll have forever and believe I'll continue to have a strong interest in this hobby for the rest of my life. That's how I justify it. Also, the used market for lenses is so deep that, frankly, if you do drop the interest in a few years, you can always sell it for probably 80-90% of its value anyway.
Again... can you really afford the hit (only you know) and how deep is your interest. If the answers are Yes and Absolutely, then why not... you live once. If not, then don't let this forum sway you the other way. This is not a "typical" sample of amateur photographers. Many here have spend $$$$$$ on stuff without batting an eye.
Anyway, that's my 2 cents.