You're basically making two pairs of trousers. There's the inner pair that should fit you as normal, though if you're planning them as over pants, they'll be a quite a bit larger than normal; and there's the outer pair, which are how much ever loft you're adding away.
I've never done this for down, but I have for batted insulation. The basic step is figure out how much loft you want (easy with batts, you don't get a choice. Say it's an inch. That will increase the diameter of things by two inches (an inch each front and back), which will increase the circumference by 2 pi, a tad over 6 1/4 inches. There's no need to add ease: that should be part of the basic, inner pair's dimensions. you can add half that to each side of the pattern pieces. At the waist and hems, you'll need to add the whole amount, or add some to the inner piece, so the edges meet. Also probably need to add a seam allowance, or change what the planned waist is. And add the baffles, etc.
You doing side zips?