I have the 100/2 ZE and really like it. I also had the 100L and liked it to. The 100L AF and IS was a plus when taking close up shots of moving subjects, e.g. butterflies. The CZ is simply different, according to tests also has higher IQ, but based on my experience both lenses are very good, but just different in the way they render and handle. I also enjoy the retro feel of handling a solid metal lens with smooth manual focus. The only thing that I have to watch out for is when taking portrait shots it can make the subjects skin look very harsh, especially adults and elderly. You either have to shoot at F/2 or have a good light, otherwise everybody will gain 20 years because of the sharpness and contrast 
Regarding manual focusing, for me MF at 100mm is much easier than 50mm or 24mm.
Never say never, but for me the CZ 100/2ZE is one of those lenses that I will probably keep my whole life.