I know more about physics than mats, but I'd say the loss by radiation is linear on the temperature difference between the two surfaces, i.e. a much colder ground will chill quicker than a warmer ground, and this is linear on temp difference.
So a high R value is reducing this radiating loss. The smaller the temp difference, e.g. warmer ground, the linear reduction of this loss, so the less the impact of the high R value. For example if the ground is the same temp the R value is irrelevant. So logically you can use high-R in heat via using heat loss from other means (breeze, no bag, thin bag, etc). FWIW I used an All Seasons in a warm September and just pushed my bag to my waist.
So theoretically the only downside of an Xtherm is your are wearing an expensive bag when you dont need the insulations so it lasts less winters. I bought a All Season regular and it packed size same as my Prolite regular so backpedalled to a Neoair short which is about half the size.