Part of my curiosity about this comes from people in the past wearing packs without hip belts and carrying quite a lot of weight. I have seen pictures of soldiers in vietnam carrying the medium alice packs without the frames and hip belt, most likely with a lot of weight (ammo, water, grenades), even cutting the belt of the frame when carrying full loads (so they can ditch the pack faster). Also, the old canvas packs were like that too, and when you consider how heavy wool blankets and cavans tarps are it makes you wonder if your shoulders can be conditioned to carry a lot of pack weight.
By the way, I am skinny and my shoulders are pretty boney, which probably makes it worse. Even a super thin hip belt helps a lot.