If you think that 4X4 is only for off-road you need to think again.
BMW X5s are not for off-road although some folks think so.
How about an Audi Quattro.
Any small saving in mpg will be at the expense of road holding in the wet, especially starting off from traffic lights in the wet where there is often a layer of oil and rubber on the road, and safety on roundabout exits.
I've driven a 4X2 Freelander and for me it's a no-no. Too much wheel-spin.
I need 4X4 every day. I can't afford to sacrifice safety.