I sleep similar to you, except my arm is on top of the pillow. I generally prefer a bag, but I run into the same issue with quilts. With a bag, I'm thinking of going to a long size to allow me to sink deeper into the bag. Otherwise, I wear a hat and my insulating jacket to bed, which generally keeps me warm enough.
I'm also able to sleep on my side sometimes with my arms mummy style and a shoulder-depth pillow.