Are you a side or back/stomach sleeper? At temps below 45* I cannot use a quilt at all as I am a side sleeper and skinny. Even with a quilt that has straps, I fully cinch them up and that does not help as there are too many gaps where there are no straps. Back/stomach sleepers have an advantage here since they have more body area covering the quilt gaps.
I have found like Bill that closing the quilt or using a top bag solves the issue. Instead of making an non-insulated bottom, maybe try instead just piecing in a 1/4" foam wedge about 17 to 12 in. wide. (17 if you are a back sleeper, 12 for side). For a recent implementation of this common technique, see the JRB Down to Earth Pad Converter for ideas. Of course, this wedge would not be your main ground pad/insulation, but will allow you to reduce that proportionately.
As for the bivy, I don't think it is necessary if you close the quilt, unless you are using it for shelter purposes. For the weight with a closed bag I would add more down instead, again only if the bivy is not doing shelter duty as well.
Finally, you probably know this already, but with a quilt if you don't fluff and position all the down correctly before bedding down, you will get cold spots. I hold both edges together and fluff all the down into the center of the quilt, then even it out a bit only, since the down will migrate from the center from body movement at night.
Hope that helps.