My experience is, leave it in. I don't manually focus often but I do find having the matte screen let's me see where I am actually AF'ing far more accurately. My keeper rate, shooting concerts, has gone up, noticeably since I started using the matte screen. Before that, I would sometimes end up AF'ing on a guitar neck or somethig other than the face and not notice. I can see that now and adjust.
With that said, AF confirm chips are crap, imo. If you are going to be shooting at f2.8 or slower, then, yes, they will sort of work. With fast lenses, wide open, they almost guarantee an OOF shot. The tolerance level seems to be very high in the 2 chipped adapters (different sources) that I have.