I.S. is extremely helpful on lenses 300mm and up, practically all the time.
I use 300/2.8 I.S. and 500/4 I.S. on tripod with gimbal mount (Wimberley Sidekick) where you don't lock down the tilt or pan settings, but track subjects with it. I.S. helps a lot. I don't think I've ever switched I.S. off, and seldom bother switching it to "panning mode".
Can't tell you the number of times I've shot with these lenses resting on a bean bag, poking out a car window, lying across on a car roof/hood, laying prone on the ground, etc. I.S. helps in all these cases, too.
Also helps on monopod, as mentioned.
Even on a solid tripod, locked down, wind can and does effect the big lenses, mostly due to their large, deep hoods.
Yes, both 300/2.8 and 500/4 can be hand held, at least briefly.
Just my opinion... I.S. is the best thing since sliced bread!