I was there...
Couple of days ago I sold the 40mm STM because it didn't give me, as a crop camera owner, anything to my photography. The lens just didn't give me a feeling "use me". Wonderful pics but not my thing.
Went through all these possibilities. Sigma 35mm A was interesting, but realistically, it would not have been wider enough. Same thing with the Canon 35, which I would have taken over the Sigma because of the IS.
Then I went through the Sigma 18-35 1.8. Tested it, and it felt awful in my hand. I wasn't about to buy a TELE, so, it just doesn't feel right with this focal length. Wonderful lens, and I was ready to put the money on the counter, but, but...
I turned to Sigma 30mm A, and found it was one of the most desirable lens in my hand. The look and the feel just yells to be taken to photography adventure. But, quite awful corner performance is something that cannot be forgiven to Sigma, when they released a new version. WHY, I ask. Remains unanswered. They just failed. Kinda okay lens from the rendering point, having quite smooth bokeh which is probably the only point why that lens exists. Awful lot of trouble with focusing, I read. No way I would buy this as I can get excellent bokeh from my other lenses.
Then I came again to the Canon IS trio, 24mm, 28mm and 35 mm. Only 35mm would give somekind of bokeh, but all three would give very very good IQ, sharpness all over, reliable focusing and IS. 28mm was my decision. It is about the best "normal" focal lenght for crop cameras.
Some comments about the 28mm IS.
-Sharp, sharp, did I mention: sharp
-Acceptable bokeh when ever you encounter it
-good balance between wide lenses and tele lenses
-very good close focusing, extends the usability of the lens a lot
-IS wonderful. I don't even think about it, but the view is steady, and the IS starts it's job quicker than you expect it to work
-feels very good quality
This lens (28mm IS) isn't anything that makes a cult following it. Nothing fancy. It just does it's job, period. I would certainly choose the 28mm IS for paid jobs over other similar focal lenght lenses.
Hope my findings help others. My idea with the choice of 28mm IS is that it is about the same focal lenght for crop sensors than 40mm is for full frame sensors.
Thank you for a very interesting and informative post first of all, as I am in the same position as you were.
The 2 main reason I want to replace the 40mm f/2.8 is that the focal lenght is bit too long as a general purpose prime lens, and it doesn't let in enough light with f/2.8.
The 28mm f/2.8 IS USM has more or less perfect focal lenght as a general walk around prime lens for me, but like the 40mm it's just f/2.8.
A prime should be fast IMO, and when there's several zooms with f/2.8, I don't consider f/2.8 to be pretty slow for a prime.
The Canon 35mm f/2.0 IS USM, is 5mm longer but f/2.0 vs f/2.8 is a big advantage. If the Canon 28mm f/2.8 IS USM is noticeable sharper and has faster AF than the 35 it might be worth it, but I seriously doubt it is.
Another lens I consider is the Canon 28 mm f/1.8 USM. But I heard it's pretty soft at f/1.8, so that's again makes the 35mm more attractive, especially since it has IS