my experience from dslr to mirrorless has been great.
i started with a canon rebel and from there i went to bigger bodies. 50d and nikon d800. as much as i love the handling and size, the weight and size of it was a con. lots of times i did not to take it with me and when i finally went on a vacation I told myself 'I am so going to get a smaller camera when I get back.' at that time I was thinking of the sony crop e-mount and a buddy told me that sony was releasing a full frame mirrorless so i anxiously waited for that. since then, i've been really happy.
for what i carried in the past as a walk around, a nikon d800 and 50 1.8, I could now carry a ful frame mirrless with 3 lens and felt lighter than the dslr+lens. i enjoyed my trip more not having to feel the camera sinking into my shoulder as I carry them around.
my old kit includes the following: a7R1, 24-70 f4, 35 2.8, 55 1.8
Ive now expanded and added a7R3, 24-70 2.8, 70-200 2.8 which causes some concerns.
the a7R3 is bigger and heavier body compared to the original but still smaller and lighter than a dslr. The 2.8 zoom lens is heavy and comparable to a dslr size lens. with this kind of setup, the thought of having a smaller and lighter set up with a mirrorless does not make sense.
i've told myself before that If i was going to get a big and heavy lens I would prefer it on a dslr and look into getting a dslr again. at the end I wanted to keep it simple and stick to one system. of course if money wasn't an issue, id have both.
As you can see I sort of went backwards to what I wanted in a mirrorless.
I'm not trying to sell you on sony here, i'm only using it because that's what I've experience with. I never owned a Fuji but i see that it's a good camera and I love the colors coming out of Fuji.
to sum up what ive said. for a light setup, use the smaller lens. if you get a bigger lens, there's no point in going smaller. i really enjoyed my original mirrorless setup, it was light and enough for what i need. i only got the 2.8 zooms because of weddings.
there's no doubt that full frame has better iso. but EverydayGetaway here dropped full frame and went with Fuji crop seeing that the iso holds up very well with not much of a difference. And from what I see, i agree.