This may not be a popular opinion here, but because I was in exactly your boat not long ago, I'll share my opinion and experience. I mainly shoot landscapes and skyscapes outdoors and use ultra-wide most of the time.
2008-2012: I shot with the Canon 10-22 on crop and loved it to death.
Early 2013: decided I had to go full-frame, so I got a 6D. Got the 17-40L thinking it would be a rough equivalent to the 10-22. Immediately realized the corners are bad. I can't overstate this enough. I was never much of a pixel peeper until I had to deal with the 17-40L on full frame. Only the center 30-40% of the image is sharp, especially at 17mm. Even halfway between the center and corner is a noticeable dropoff, and 75% of the way to the corner is mush. At f/8.
Now: I've switched to Nikon, in part because I concluded that Canon simply doesn't make any FF ultra-wide zooms that give me IQ on par with what the 10-22 did on crop. Even the 16-35L II, despite costing a ton, is bad in the corners. It's really a shame.
If ultra-wide is your bread and butter, and you're averse to using primes, I'd seriously consider either sticking with crop or switching brands. If you don't mind primes, a combination of the Samyang 14mm and perhaps the Zeiss 21mm could work well, but that's a chunk of change and not very convenient. The Nikon 14-24mm can be adapted for Canon as well, but again, pricy, heavy and doesn't take filters.