Seeing that you already have the Thermawrap, I would add a vest. I have a Thermawrap that fits fairly snug and a Patagonia micro puff vest that fits over the Thermawrap. It's not unlike a 3-in-1 jacket where you can wear one, the other or all.
In general, more light layers are going to be more adaptable over a greater range of conditions. As others said, one monolithic layer is fine for sitting still, but just dead weight when you are moving. An ultralight selection of clothing is just right when you are wearing all your layers in the coldest conditions you encounter--- and you are comfortable. In a similar vein, I am a proponent of wearing more to sleep in vs. a heavier sleeping bag. I think you could split hairs forever over how layers may absorb moisture, etc. If you are in camp, it shouldn't be a problem and I would have to be in sub zero conditions to wear two insulation layers under way.
The other trick to feeling more comfortable is to take care of your extremities-- good gloves, hat, and socks take the chill off for me.