I have both lenses, and still sometimes use the 35F2 for travel. Both lenses are sharp, but there are some advantages to the newer IS version that make the additional cost worth it.
Flare resistance is better on the new lens. The older lens has monstrous purple flare if used without the hood. The newer lens is much better - but I generally use the hoods on these lenses. I learned quickly that it was needed on the older lens.
If you are not at F2 on the older model, you get strange bokeh. Some people like this, but it is much smoother on the new lens.
AF is almost silent on the new lens, compared with the older model and faster. I think AF is fine on the older lens, but it is superior on the IS version. Some people complain about the noise, but it has never bothered me. My hearing is lousy. For candids, the USM AF is worth it.
If you can see it, you can photograph it with the newer IS lens.
Color and contrast are different. It is not simple to say which one has better color, but contrast is higher on the newer lens.
I think that build quality is better on the new lens. I use this focal length heavily, and had to replace the aperture mechanism on the older lens. I can't tell you how many exposures were taken before this was needed, but it was a lot. This is something to consider if buying used.
If using a crop camera or full frame, the newer lens is a better choice. You can certainly get sharp photos with the older lens, and it is lighter and smaller. I am keeping it for backup. I don't think you can go wrong with the IS version, and IS does help with getting sharp photos.