When only securing the glass filter in front of the camera lens with rubber bands you're also allowing light in from the backside of the glass that will create reflection within the glass. In addition to taping the edges you should preventing light from getting to the exposed backside of the filter glass that encircles the camera lens. That should help considerably as well.
I made a simple lightweight holder made from layers of 3/16" plywood. It hooks onto the end of my camera lens and allows me to slide the glass filter in and out. No light leaks or reflections