I wear most of my hiking gear every day. When I have to buy socks, I buy the type I like to hike in. Same with underwear... too much info. I'm a tshirt and jeans guy, so the shirts transfer over, although I've never worn a long sleeve in every day wear. Shorts get worn on summer hikes and summer everything else, and for running year round. I wear the same wool hat on cool winter days that I take backpacking all year. Same with sunglasses, baseball cap, and shoes.
I own a button up Columbia nylon shirt that I like for mountaineering or bug season, and I suppose if I needed a button up for some casual use, I would use it, but I never have.
Pants is where it ends for me. Regardless of the oddly poor reputation they have on this forum, I like convertible nylon pants for hiking if I suspect a lot of sun exposure or bugs. Where I hike, that accounts for 3/4 of my summer. However, I never really wear them casually, I think mostly because I own 4 pair of jeans that I've had for 6 years and hope to have for 6 more (They are getting pretty patchy...).