But they don't buy them for relatively good handling. In fact, they aren't particularly good relative to anything mainstream.
Even less so for a Lotus Elise though. I had one for a weekend a few years ago. Horrible road car, so low to the ground, you suck up the exhausts of all the other vehicles, poor vision also, rock hard suspension for the potholled british roads, minimal load carrying capacity. A total swine to get in and out of, no decent sound sysytem, no Aircon for swealtering days sat in traffic...etc . In fact they are less practical than my 680bhp per tonne bike...at least you don't have to sit in traffic jams on that thing sucking on someone elses tail pipe
My 4x4 was bought specifically because it was heavy enough, it has a book speed of 82mph, and 0-60 in about 30 seconds (even in regular 2wd mode). I don't see the point in an X5 as they are not designed to go offroad at all, but if money was no object, I'd have a V10 Toerag for the 600ft lbs of torque

(but still stick All terrains on it as it has a decent gearbox)