Here are my thoughts on mirrorless being a replacement for DSLR, or being used for smaller everyday carry around cameras.
For me mirrorless is a replacement for traveling and street stuff only. I will not travel around the world with just one body, so when I use to carry my Canon gear it was beyond bulky. 2 bodies with a few lenses meant carrying a backpack instead of a smaller shoulder bag. In some of the places I traveled in Asia this became an issue because the smaller airlines always wanted to weigh my backpack, only to find it too heavy and would try to force me to check it. Needless to say when I finally decided to stop traveling with my Canon stuff I was much happier because of the lesser weight and no hassle from airports.
Heres an example of using smaller gear and what can be carried at a fraction of the size of a DSLR. I went a little over board on one trip to the Philippines and Hong Kong with the gear I carried. My last trip consisted of an M9, M6, 35 cron , 50 cron, and voightlander 21. Along with that I also carried the X-Pro with 14, 35, 18-55, and the x100s. The shoulder bag was jam packed with extra batteries, filters for B&W with m6, film, memory cards, and so on. Even with this large kit it was not even close to what the Canon kit would weigh and never felt uncomfortable on my shoulder. I never carried all of this at one time when out shooting, but for transient purposes it was still much easier than hauling the DSLR stuff.
With that said, I just spent little under a week walking the streets of NYC with an E-P5. I started out with the 12 and 17, only to pick up the 45 when I stopped in at B+H. I could comfortably put one lens in each cargo pocket of my shorts, along with a spare battery and memory card in another pocket and walk for hours. I'm confident that if somebody needed a camera to take everywhere than this is surely one of those. So combine it with what looks to be an awesome camera in the E-M1 and I think my travel kit will finally be figured out. I felt the files were pretty good considering the ISO I was shooting at most of the time and felt the EVF was great, but also the LCD was pretty handy at times. Will the E-P5, or the new E-M1 slip into a small pocket so you can have a camera with you 24/7? Probably not, but to me thats one of the trade offs of having a small, yet very flexible setup capable of interchangeable lenses. My x100s will fit in my front pocket without hood installed, so I may keep it just for that when I don't feel the desire to carry anything else. Also, the Ricoh GR, X20, or RX100 could probably fit the role of pocket able cameras.
Anyways, sorry for the long winded post, but thought I would share my experience and thoughts on how useful a smaller setup can be when compared to DSLR gear. Like I said, the Olympus, or any other gear will not totally replace my DSLR gear, but will compliment it very nicely.