The common design changes I see are that the shoulder straps attach further in on the pack to accommodate a woman's naturally narrower shoulders, S-shaped shoulder straps to fit around the breasts instead of going over the top of them (not on all women's packs) and a smaller waist belt, often positioned differently than on a man's pack to fit a woman's hips.
But in some cases, a "women's" version just means it's the man's version shrunk down a little with no real changes to the design.
Could a guy wear it? Sure, if it was large enough to fit him. Whether he would find it comfortable, that I couldn't tell you.