Jonathan, 2 ideas come to mind. If you are using an inflatable pad, like a Thermarest, you can insert it like a pipe inside of the pack. You do this with the valve open, then after you pack your stuff into the void, you blow air into the mattress to make it firm and then close the valve.
The other idea is a corrugated plastic sheet cut to fit. The same stuff used for signs. If you need to curve it to fit your back you can insert some wire into a few of the corrugations to give it some shape.
The first idea is something I do in my Jam. The second idea is one I read about on the forums, but have never tried. You could also sew a couple of channels for aluminum stays, I suppose.