Personally, I would remove each from their frames. Time consuming? Sure, but that's the right way to do it. Ultimately it's up to you though.
Technically, though, if you watch your angles, you shouldn't get glare from the lights. Your biggest issue, I think, is avoiding shadows from the frames and mats. If you keep your light back and to the sides you should be able to light the artwork evenly. As for reflections, I've had really good luck shooting through a piece of black foam core with little to no reflections visible. And you don't need a totally dark room either. And yes, you can (should) shoot perpendicular to the artwork. That will save you hours in post processing.
Since you're shooting framed pieces, consider rigging up a frame holder similar to an artist's easel up on a wall. This will allow you to easily adjust each frame to center on the camera. You can then move your tripod closer or further as size demands without constantly having to adjust camera height.
Just some suggestions.