Glass first, but even that has to be done with some care.
For instance, you have the 18-55 IS and the 55-250 IS. Both of those lenses produce quite decent image quality.
You're looking at the 24-105L, but the real question is: what advantages would you gain by getting that lens?
Your current lenses already have good IS, so you won't gain anything there to speak of. Your current lenses range from f/3.5 to f/5.6 (depending on focal length). The 24-105L gets you f/4 across its zoom range, but that's at most only a stop faster than the current lenses you have. And your body has no weathersealing, so the weathersealing on the 24-105 won't be of any use to you, either.
The bottom line is that you won't be gaining all that much by going for the 24-105L.
Just as with your camera body, you should shoot for a while with the body and lenses you have until you figure out what you're really missing. For instance, you may find that you want a significantly shallower depth of field, and therefore need a significantly wider aperture for some shots. For dealing with that, you may want to start looking at some primes. There are some very good ones out there, e.g. the Canon 85 f/1.8, the 50 f/1.4 (I prefer the Sigma version of that one, though, but the Canon version is no slouch), the Sigma 30 f/1.4, etc. You can pick up two or three of those for the price of just the 24-105L alone.
Or you may find that even 250mm isn't long enough for the shots you're taking, and you need something longer. There are quite a lot of options there.
Or maybe you decide you can't focus close enough and need a lens that's capable of doing macro. There are lots of options there as well.
But until you've got a decent amount of experience, there's no way you'll really know what you need, and you'll wind up putting your money into lenses that you will gain little from.
Oh, one more thing: pick up the Canon 50 f/1.8. It's cheap enough and good enough that it's worth having just so you'll have a prime lens to play with, so you can learn the ins and outs of wide-aperture shooting.