Sounds like my family's trips. We did a 3-nighter with mom carrying the childcarrier, another obviously on the way (6 months pregnant) and I carried everything else, which was pretty much everything. It was an era of cabin trips so as to avoid the
Currently, our 7-year-old can take her snacks, outer clothes and stuffed animal. Our 12-year-old can start to take some weight and more so, some of the volume, but not all of his own gear (a big motivation of mine to switch to more UL gear and techniques).
That leaves me on the father-son trip and my wife and I on the family trips to split most of the gear and food for 4 people between us.
I use a Kelty framed pack with large stuff sacks high and low. Heavy stuff goes in the main pack bag and a few sleeping bags and sleeping pads go into each large stuff sack.
My 12-year gets my Jam 35 or Jam 70 at about 12-15 pounds and as much volume as I can shift his way.
I'm at about 45 pounds and my wife at 35-40.
In answer to your original Q, it depends on how far you're going. If you're only going in 3 miles, you can carry anything for 2 hours (and, yes, it will take you 2-3 hours to do 3 miles with your crew). If it longer than that, I'd go with a comfortable framed pack, use as much UL gear and techniques as you can to keep the weight down, but accept that a traditional pack is in some ways better for heavier loads.