You can test by removing the gel from the filter and replacing the empty filter, or by "leaving it open" if this bothers you (dust) some gaf tape would leave no residue, and last perhaps years, definitively months.
On my 500mm, it certainly appears to work just fine with no filter in the optical path, but the af distance shifts (if I recall correctly) The ie: you still get an in focus shot, but when you remove the filter, it will re-focus to get back in focus.
I was wrong about the "stock" item that came with this generation lens. The supplied filter is always the "gel holder" drop in, BUT they are supposed to have the same clear glass filter already installed in the gel holder.
On both my 500mm and 200mm the stock "gel filter holder" has a clear glass plate installed. The Gel I think it supposed to go in with it, not instead of the glass.
Does your gel drop in holder have the glass? If not, I'd suggest getting yourself one of the clear glass filters.
If there is a gel in there you can remove it.