Sorry no experience with using buttons on a down vest here.
I do use plastic Kamsnaps on my top quilt and they seem to hold their fastening strength pretty well so far.
Have you considered another option like omni tape. It's a velcro style product with both hooks and loops on both pieces of the "tape" so that there is no snagging.
Quest Outfitters has a version of it that I have used on a pack. The link below will take you to a point about 60% down towards the bottom on a Quest webpage. Scroll down until you are about 75% to 80% towards the bottom and look for "HOOK & LOOP- SNAG FREE FASTENER -like Omni Tape®".
http://www.questoutfitters.com/narrow_roll_goods.htm#HOOK & LOOP
Potential problems? Velcro is noisy. ;-(
Potential problems with buttons? Sewing one back on with cold fingers or threading a needle with cold fingers and eyes as old as mine. ;-)
One more suggestion if you stick with the buttons, use an easily spotted color if you have to find one, "that has left the building", on the ground. :-)
Post pictures when you're done please. You know how us MYOG types like pictures.