Depends on how you approach it.
I have several APS-C cameras and a full frame. My zooms are on the extreme ends, ultrawide (11mm) and telephoto (600mm). Everything in the middle is a prime. I don't need zooms to cover focal ranges. I'm either shooting 35mm or 85mm usually. Sometimes I use a 40mm and/or 50mm depending on need.
It really depends on what you shoot. Personally I do not compromise mid-range focal length zooms for their aperture. Shooting at F5.6 or even more narrow is not my cup of tea at 85mm~135mm, APS-C or not. I'll shoot F8 from 600mm. And at ultrawide to wide, I'm ok stopping down to F8, F11, F16, etc. But from 28mm to 85mm, I don't want anything less than F2 as an option.
I have EOS-M instead of SL1 for my `small' interchangeable APS-C. I also have the 650D and XSi. But I use them for different things. The EOS-M is my travel/every day quick compact. And I only need one lens on it, the 22mm F2. No other lens needed at all. I do anything from landscape to portrait with it. And that's all I need from a compact.
If I were building an APS-C `light weight' setup right now from the ground up that had a viewfinder, I'd probably be doing this:
EF 28mm F1.8 (every day use, for every thing)
EF 40mm F2.8 (for travel/out and about when you want lowest profile possible; it's cheap)
EF 85mm F1.8 (specific for outdoor portrait and sports)
Yongnuo 568EX II with YN622's