I have both lenses. Used the 55-250 for a year or so, the 70-300 for 4 months. Both on a Rebel XSI.
Shot at airshows mostly, ranging from scale RC aircraft to the real deal.
The image quality is VERY similar, differences are seen mostly at wide open apertures with the 70-300 being slightly sharper (i needed that extra sharpness, I like to crop and need to more often than not).
At f/8 they're almost impossible to distinguish.
The USM motor on the 70-300 is not much faster than the 250's. But i did see a difference (more keepers seems to be a good measure for that).
Not that the 250 is bad, it worked pretty good shooting RC aircraft but I've found myself using AF less, because it failed too much tracking the planes even when close to infinity.
The 70-300 did better with locking AF accurately at all distances.
The extra 50mm were useful many times, allowing me to crop less. Every bit helps.
Regarding build quality- both are not even up to my Tamron 17-50.
The 250 always felt like a toy to me. The zoom ring was hard to move accurately- it would always jump a bit when moving it slowly. The 300's zoom and focus rings feel and operate much better, smooth and with no zoom creep (250's barrel will drop like a rock when pointed down).
Also, the 250 is a great dust pump! Not only are some internal elements are covered with nice dust (that doesn't seem to impact IQ, but it's still there) after a year now, but I've also noticed that less dust made it onto my sensor with the 70-300. I don't see any dust inside the 70-300 yet.
The 70-300 is noticeably heavier and larger than the 55-250.
the IS on the 70-300 seems a little more effective, maybe it's also because of the weight.
The IS also was VERY useful at 300mm in many cases, especially in mode 2 for panning.
I like the 70-300 better because it's built better, keeps my sensor cleaner and has that little edge on sharpness and reach. The price really is rather high but it's an EF lens, more glass and all.
I bought mine second hand, still great condition, so the price was a little lower.
If you really need a compact lens- go for the 55-250. If you need some more IQ and performance, the 70-300 will help.