Yeah, Micro 4/3 has some great primes. See how you go with the 20mm; there have been a few issues reported with the E-M5/20mm combination.
The first is the possibility of high ISO banding caused by some electronic interference between the lens and the sensor. Whatever the problem is, it has only been reported with the 20mm lens so far.
The second is occasional camera lock-up where the battery has to be removed and re-inserted to wake the camera up again. I had it happen for the first time today, but hadn't had any problems previously.
So far my favourite lens on the E-M5 is the PL25/1.4, although to be honest that is my favourite lens on any Micro 4/3 camera .