Phone cameras are great because they're always there but I never get anything I'm that happy out of it. That's not to say they aren't getting better, I'm impressed with how much better my friend's Note 3 is than my Note 2.
That said, while I use my phone to record snaps of things I'm working on all the time, I also keep a little Canon Elph around because of the zoom range, better image quality and stabilization because if I'm tearing something apart I can get a better shot of things as I'm working on them with it than I can with my phone in many cases.
When it comes to actually enjoying what I'm shooting with, which is something I always appreciate, I've not yet used a P&S that provides that. I would still rather break out an SLR or other camera with better controls. I'm sure something like the Fuji X20 could get me there but I'm going to give the EOS M a shot for now. It misses out on a lot of controls and I don't know how I'll feel about the touch screen, but the sensor & compatibility with my other gear in a pinch sounds hard to pass up for the current price.