I'm of the same opinion as the previous poster. I try not to carry anything that can't serve a dual purpose. For that reason, I wear the thinnest synthetic long handled underwear I own as sleep wear. That way, I can wear it around camp in the morning while I air my bag and make breakfast. By wearing it only to bed after wiping some loose grime from myself, and cleaning the nether regions, I have something I can wear into town, if necessary (the bottoms being worn under shorts, of course).
Since you asked for solutions regarding three season hiking, I would not suggest using a vapor barrier. They tend to make you overheat and sweat in temps. over 25F or so. Also, anything you wear inside a vapor barrier gets very damp, so don't wear insulating layers in one if you use one in the winter.