If you're just using/carrying the one lens on the body or the body and kit lens plus one other small lens, you could probably use the Osprey Grab Bag. It's like an oversized hip belt pocket that instead attaches to your pack's shoulder straps and is in worn over your upper abdomen slightly off to the side.
It stays out of the way of my line of sight to my feet and holds my Sony NEX plus two small lenses. If padding is needed, you could probably work up an "inner" sleeve to go inside the grab bag using bubble wrap or some thin closed cell foam.
If you want more room for extra lenses and/or gear, take a look at the Zpacks Multipack. It's a bit larger and can be rigged a number of different ways. When I'm on a photo-oriented trip, I bring this bag and can fit my camera body plus 3 or 4 small lenses in there and wear it as a chest harness. Again, folks have made their own "after market" padding to fit inside the bag, if needed.
Having the camera accessible makes more sense to me than packing it away in a pack where it becomes more of a burden to stop, take off the pack, open up the pack, open up the camera bag, take picture and then re-pack before you can get moving again.