Given that even the name brand laptops have been having issues with their OEM batteries catching fire, I'm not sure I'd go with an off brand. Plus, in almost all laptops I've seen, the battery is part of your laptop's external casing and appearance, so fit and finish may not match up. Because of that and the rapid model turnover, I don't think there's actually a big market for aftermarket batteries on laptops. I've seen some, but it probably isn't as profitable as it is on cameras and cell phones where you don't have to match the OEM battery's appearance perfectly. Plus a laptop battery has to provide a lot more juice, and therefore is probably a lot more complex.
You're absolutely right though...once your laptop battery no longer holds a charge, you often are faced with the prospect of paying a significant percentage of the cost of a new laptop just to get a fresh battery. You're still obviously spending a lot more to upgrade the whole thing, but it does often feel like a better use of cash. If you can afford it and if you feel like your existing laptop isn't meeting your needs, I'd say go for the upgrade. If you can't afford it or you think it's fast enough that you'd use it for a few more years, then I'd replace the battery.
You could also try ebay for the dell branded one, but of course with a battery you want it to be new or very low usage, and you're taking their word for it there.