Girls are different than boys. The girls may tolerate backpacking, but the boys may love it.
When the kids are toddlers, think about car camping. We had a pickup camper, and it was great. We did fishing, bike riding, and short hikes.
When my son was about 6, we walked a mile to a mountain lake and camped. He carried a small day pack, with his lunch and a rain coat in it. Each year after that we went on longer and harder backpacks, maybe 6 per year.
When he was 11 we joined scouts. I say "we" because I committed to be involved, lead trips, take training, wear the uniform, the works. From that point on, he wasn't going on a hike with his boring old Dad, he was going with his friends. We didn't particularly like the culture in our troop, so we changed it by leading backpacks, and training the new scouts. When my daughter was about 15, she swore off backpacking, FOREVER (so far).
When my son was old enough I led a 4 day backpack for younger scouts, and last year an 8 day 50 miler. He was 13. This summer, we are going to the Wind Rivers on a 50 miler.