I first had the non-IS. That was fine for daytime shooting around the parks, birds, action shots, portraits, etc., as long as there is sufficient light. Once the sun starts going down, the non-IS becomes very difficult to shoot with, even if you're shooting non-moving subjects. I usually have to keep a shutter speed of at least 1/160 to 1/200 to get acceptable (not 100% sharp) shots of still subjects; even faster speeds if it is moving (duh).
I find that the IS makes the lens much more versatile for every day use from moving subjects to portraits, daytime or night time. I can shoot still subjects at 1/30 with the IS on. That's a tremendous difference. the IS even helps me in studio shots since the maximum trigger sync speed on my RF-602s is 1/200. I can rest assured that there will be no risk of camera shake when I take studio portraits.
My advice: buy a used non-IS first and see how you like it. At the same time, start saving up for the IS version or another lens. If you think you'll need the IS version, then sell yours and buy it. That's how I experienced both lenses.