I purchased a Smartwool 1/4 zip last winter and when I pulled it out to use in May a moth had made a few hole in the front.
My Grandmother and Mother had taught me how to Darn wool socks so I went to the fabric store and picked out some thread that matched the fabric and darned the holes. You have to look close to see the darning.
The trick is not to pull the tread tight, in fact when you start leave the tread loose for the first while, it will get taught by the weaving pattern of the additional thread. The mistake I make is pulling the tread to taught. It is darning, not sewing, you are not trying to close the hole by pulling the fabric together you are filling the hole with thread.
For socks my mom would put a light bulb in the heel (usually the first place for a hole) and then darn the sock. This keeps and even tension on the sock and saves you finger while you are darning, it also keeps the shape of the heel.