I've had both (currently I only have the 5DIII and a 70D), and I have to say I personally wish the 5DIII was housed in the 6D body. To me the 6D felt much more comfortable and better contoured, but I don't have large hands. And the sensor of the 6D is a bit better too, any time I have to shoot ISO 6400 and higher, I wish I was using the 6D's sensor.
I never ever felt that the 6D was built like a toy. I've had other 5Ds and even a 1DII and still the 6D felt like a well-built camera in my hand. It's lighter and smaller than your average full framer and that may trick people into believing it's inferiorly built.