"But how about we handicap mileage? It's just not fair that those tall folks take fewer steps to hike a mile!"
Elite marathoners are usually not tall. NBA championship winners tend to be tall.
Everyone can win at backpacking even if you are tall, short, round, thin, red, yellow or green. And no one will measure your pack at the end of the day... except maybe yourself.