I'm in Oklahoma. Our summers are quite hot and for years I used a vest in the heat. Now I use a belt. I bought a simple web belt from Lowes ($5) and I put Tamarac 4 pouches on it, two for extra lenses, one for filters and one for other stuff. My camera rides on my tripod but could as easily be carried on a strap of some sort. It beats vests and shoulder bags. My backpack just gets all of the gear to the site. Then the belt comes out. It's light, comfortable, cheap, and doesn't get dripping wet in the summer. If you are not taking much stuff (digital requires much less than film did) consider a belt system of some type. There are some dedicated photo types or the made up kind I'm using.
One important thing: Don't use the belt that holds your pants up. You'll feel like they are about to fall off and it's very uncomfortable. Use a second belt which has the sole purpose of carrying your gear.