okay, i just went and did it. however, i left a shorter, intact ring of the rubber so i could get it back on, easily if i wanted.
i could have taken the whole thing off like a bike grip, but this was easier and i can still put it back on:
previous weight: 2.1 oz
new weight: 1.8oz
meh on the weight, but it makes the whole thing smaller in my ditties bag.
it seems to stay tucked under the webbing on my shoulder strap just as well as before...