I have shot similar focal length lenses, 300L F4 IS and 400L F5.6 + converter, before jumping to the 500L. The difference is night and day. With your 100-400, you can just take a stroke in the park and point your lens at will whenever you see some activities. With the 500L, you can't. That's because the size and the weight of the lens. Also, the focal length gives you a very narrow angle of view. Is really hard to use it to spot your subjects. Even if you could, you would have a hard time keeping it steady and placing the AF point on the subject, without causing the body to AF something else. A slight movement to the right or left, you can misplace the AF point. When the AI servo is engaged, the AF system will change focus.
I did it once mainly shooting handheld in Central Park. It was a very tiring experience. I like to be more mobile and I use a monopod and a full gimbal head to support the 500L. The flash is mounted on the hot shoe. I could still hike for hours with it and less tiring. Because the minimum focus distance of the lens is something like 9ft, it is very, very difficult to use it in a dense vegetation area.
Because of the excellent optics and the reach you get, you get more good shots with it than with your 100-400. You don't need to be that close anymore. A 50% crop shot from the lens is still usable and you could hardly tell the difference.