If I'm going for full-on desert use (e.g. 2 weeks rafting the Grand Canyon), I've got a goofy, nylon, very broad-brimmed hat with chin-strap to keep it on in the wind. One is by Columbia and another is by OP. But if I was more active - thru-hiking in the desert, I'd go for a silvered umbrella.
For UL backpacking, I'd use a visor and put my silk bandana under it. And in that mode, the bandana covers as much of my neck, ears, and sides of my face as I want. It isn't as cool as the broad-brimmed hat, so I think of it more for high-Sierra use where the it is cooler than in the lowlands. The visor is light and cheap. The silk bandana is 36" x 36" and multipurpose - neck scarf in a cold wind. Blow my nose. Bank-robber style facemask in a sub-freezing wind, etc.
I got a few of those 36x36 silk badana on Sierra Trading Post a few years ago for about $9 each. I'm sure they don't have them anymore. Sometimes you can find big silk scarves at a thrift store (but always in truly funky, white-trash patterns).
For hot travel with water sources (e.g. main trail corridor in the Grand Canyon), I like a cotton bandana and/or cotton, brimmed hat instead because it carries more water when I soak it at every stream crossing and drinking water source. I find water I put on the outside of my body cools me without being as tiring as when I sweat that same amount of water.