When backpacking my camera lives a bag which has been attached to my shoulder strap. Most recently, it is a lowepro lens case which is a perfect size for an Oympus OM-D + pancake lens. Over the years it's been a variety of things include an Eagle Creek utility pouch and just a light ripstop bag without any padding - how often do you need protection at a mid-chest height. Extra lens are normally in a side pocket in a domke wrap for padding which goes inside in dry bag when it's wet.
When not backpacking I use a domke f-5xb which has room for body with a lens attached (except for longer zoom), several lens, filters, extra battery, extra memory card. Depending on what I am doing this is attaches my a waist strap, used with it's over the shoulder strap, or put in the lower section of a divided daypack with the top flap open so I just unzip the daypack and call immediate take out or put in equipment (basically use it as a protective pad).