Back when Toyota Corollas were 2500 pounds, I used one to get to an awful lot of trailheads. With 4-5 people along, the people-miles/gallon was incredible and when I'd get stuck, we could just push it out.
I still mostly see 4WDs used by people who think it lets them go faster in bad conditions (true, that) but without consider that nothing about having 3 differentials helps you stop any faster than 2WD. A shovel, come-a-long, tire chains, and experience will get one through more than a Hummer will. I've gone over mountain passes on the Arctic Circle in blizzards only to surprise the 4WDs at the other end of a "closed" highway in said 2WD Corolla. Yeah, I had to shovel some 3-foot-high snow drifts, but somehow it's more fun shoveling a state highway than your own driveway.
While I mostly see 4WD used by posers and paranoid people, I'll say that a road trip on Mexico 1 in summer, the Alcan in winter, or the Dempster Highway anytime has some of the same feeling as extreme backpacking - being out in the middle of nowhere, somewhat on your own, and having to keep your wits about you.