Sounds like too much clothing or the wrong clothing.
Alpine skiing and hiking with skis , I wear only thin polypro top , light synthetic shirt, and uninsulated shell, even with single digit temps and high winds on peaks and exposed areas and never have been cold. (nose,ears, fingers, lips excluded) Lower half is polypro bottoms and lightly insulated ski pants.
My clothing is never "wet", not even moist at end of day it dries so quickly. Wearing other clothing in the past it did get soaked though, so I think the choice of clothing is important for moisture transport.
If I start getting warm, the jacket is unzipped and so are the pit zips until I get cool again. And gloves/mittens come off. If Im too warm, off comes something else, shirt or polypro top. If Im cold, things get buttoned up, and neck gaiter gets added.
I used to carry extra layers with me in a pack, but never needed them