I keep my camera attached to my shoulder strap with a Peak Design Capture Pro Camera Clip (though I think the cheaper Capture Camera Clip is lighter, if less durable): https://peakdesignltd.com/store/?c=clips
A camera in my pack is a camera that doesn't get used!
When I know that I won't be using the camera, then I put it in a dry bag (currently, a Sea-to-Summit roll-top dry sack) stuffed inside of my backpack.
I only bring one lens, an 18-270 Tamron, so that I don't have to worry about fidgeting with lenses (and dust in the camera) while I'm in the backcountry.
Accessories (extra batteries, memory cards, lens wipe) go in a small zippered bag in a front pocket of my backpack.