Ultimately, like so many things, there's no one right answery and it depends on the business you want to build for yourself. For me, this is a part time gig and will continue to be until I'm ready to retire from the "day job" in about five years. So until then, I plan on keeping the business to 12-15 weddings/year.
Because it's a part time business, I run it out of my home and keep my costs very low. When I think about my fixed on-going costs, they are surprisingly small. There's the annual cost for the business registration with the state, my liability insurance, the fees for my primary site and on-line gallery, and advertising. For me, those things don't total $1,000/year.
I know some have touched on vehicles, computer equipment, phones, etc., but for me, those aren't costs as much as they're substantial tax deductions and financial benefit. I already own two vehicles (owing to the complete impracticality of the one) and I'd have those anyway, only now I get to deduct their use. My home office was built prior to getting started and now it allows me to deduct a substantial portion of my home expenses. Just hanging out the shingle has allowed me thousands in tax deductions against costs I was already paying.
There is the matter of samples, but if you're selective and partner with the right providers, you don't need a ton and they are typically offered at a pretty steep discount; some even offer them for free.
A big cost, of course, is gear, hardware and software, etc. But that's very individual and I assume you already have much of what you need. Only now, you get to deduct for its depreciation.
But again, that's for a small home-based business. It's very easy for me to be wonderfully profitable given this model, but hard to earn what I'd need to live on if it were my only income. To make this a "real" business, I'd have to invest a lot more in the interest of driving greater revenue. Much, much more difficult and one of the reasons why I am deliberately keeping my business small.