In my experience with a 1 gb MicroDrive, and from the posts I've read over the past 6 months, I do not believe they are at all fragile.
That said, relative to Compact Flash Type I, which has no moving parts at all, they will be more susceptible to mechanical damage should any significant jarring occur (for example dropping on a hard floor from a significant height). I don't believe normal handling is likely to result in any damage. They are little tiny hard drives, with moving parts, so there is some potential - but I personally believe this is minimal (again, with reasonably careful handling).
Hope this helps.