Being a slave to your job is one thing that I'd like to avoid. Though when you are in a transitional or building phase, more effort has to go into it. And yes, you hit the nail on the head in that quality of life is more important than money.
In the last couple seasons, I haven't been shooting nearly as much, but it has actually helped me improve greatly. Before I was in 'survival' mode, just getting through all the shoots. But now, I have time to evaluate each shoot, pre-plan and think of new ideas for each upcoming shoot, and sit and brainstorm new ideas that I want to try. Shooting less has improved my photography... ironically enough. Not a good strategy for a beginner, but after shooting 13 years of weddings and 22 years of photography, it's nice to be able to sit back and really enjoy shooting again because I can put more into each shoot than I could before. And that includes post processing, as you mentioned.