Heya,
It comes down to working distance. Portrait isn't just some predefined genre that requires long focal length and really wide aperture. Some say they are doing portrait but really they're just doing family photojournalism with wide aperture to get a look they like, while also blurring out most of the unsightly uncontrolled environment. Nothing wrong with that, but, if that's what you're doing, longer and wider aperture is a good way to do it outside. But if you take that out and you consider other forms of portraiture, especially studio work, posed portrait, etc, there's just a lot more to it. 35mm is just as good as a portrait lens as 85 or 135 depending on what you're doing with it, the kind of portrait you're doing, etc. Again, portrait work isn't strictly long and fast.
So look at what you do with your 85mm. Look at how you approach portrait, and what portrait even really is for you. If you want more isolation and you want to work from even farther away from the subject, then the 135 is a great way to do that. But if you like being closer to your subject, and you want a smaller, lighter setup, 85 is a good way to go and ultimately is much more versatile as a general lens. The 135 is very, very specific and requires the working room for it to do what it does, unless you're doing strictly face or bust portrait, and even then, that is still assuming it's virtually always outside with lots of background to subject distance, which is where you see the creamy bokeh come in--otherwise, it's not going to do anything out of the ordinary.
I used to shoot 85 F1.4, and I contemplated 135 F2, but ultimately I got rid of my fast long lenses and I have 90mm F2.8 with image stabilization (macro) that I use for 99% of my portrait work outdoor, and I have a 200 F2.8L if I really want to get that telephoto look, but more and more, it's just too much working distance, so I literally use my 90 F2.8 most of all. It covers everything for me (macro, portrait, etc, and has image stabilization). I still get plenty of isolation with F2.8.
I literally use 35mm & 90mm for almost all my portrait and general family photojournalism. I don't think I could do 35 & 135 though, I like the middle ground, I don't want to stand back 20 feet or more all the time.
From a recent shoot, where I had my daughter and her cousins together. I used 35 F2 & 90 F2.8. To me, portrait isn't about just blasting out a background, it's about the feel of the mood, the environment, and really, to me, portrait is about lighting.
I put a lot more into my lighting than I worry about a lens. But I'm just a dad with a kid and a camera, this is a hobby for me.
35mm:
IMAGE LINK: https://flic.kr/p/KbUY7n 
IMG_4014mark by
Martin Wise, on Flickr
90mm:
IMAGE LINK: https://flic.kr/p/KiQiS6 IMG_7328mark by
Martin Wise, on Flickr
Very best,
