I wear my down vest when I can. It's lighter and packs up smaller. Combined with my shell, it's more versatile than an UL jacket, but probably just as warm.
I take one of my down sweaters when the temps demand it. Sometimes that means just the EB FA jacket, others my Nunatak hooded sweater.
I know "when I need it" isn't helpful, but I don't follow any rules about it. I look at the weather forecast and ask, "Do I want the extra weight and bulk of one of my sweaters, or do I just grab the vest?" I think having both to choose from is great, but plenty of folks dig their Montbell EX Lights and at that point, why would you go out and buy a vest?