No problem, it was natural to ask birders at about long lenses. And the fact is that back when "Digiscoping" was popular, about 90% of the application was used for birding
The drawbacks of not using a camera lens are multiple;
No auto focus is the most obvious.
No ability to change aperture, so all exposure changes must be done via shutter speed and ISO adjustment.
Image quality is usually very different from a dedicated lens. but of course the quality of the spotting cope/telescope will vary just as it does with camera lenses. That said, you can gt some pretty terrible image quality form spotting scopes. Hard to find a real camera lens at made at such focl lengths that will be as bad. But again, with a good spotting scope like Kowa, Pentax, or at the HIGH end like Swarovski, you can be in very good shape.
Of course the advantage to a spotting scope is cost. Starting at around $400.00 and going up to a few thousand, they are even at the high end a good deal cheaper than say, a Canon 500mm f/4