Heya,
It's all personal preference. There's no right or wrong answer here. You could use a 35mm. You could use a 14mm! Distance to subject is what effects distortion, and telephoto lenses put a lot of distance between you and the subject, and that's basically the only significant reason folk use telephoto lenses for portrait--the resulting distance. You will be hard pressed to see a significant difference between 85mm and 105mm. In terms of distance, we're talking a few inches different. That's all.
Sounds like you're doing this in a studio (back to a wall).
Why do you feel you need a prime? Why not just get a sharp zoom? If you're in a studio, you're likely not doing F1.4 shots.
You could use a 35mm... 50mm... really doesn't matter. What matters is what you want it to look like, your working distance, and other features you feel you need.
85mm is a classic focal length for portrait for a reason. It's a good working distance while keeping you at distance, not too close (distortion) and not so far away that you have no room to work.
Very best,