I just did that with a poncho. I have a small vice with rubber jaws and I was able to clamp on the edge of the grommet and the corner of the cloth. I warmed it up with a hair dryer so the fabric would soften and stretch and I could slide it off one edge of the grommet.
It was an old poncho, so I wasn't worried about "pretty" but it wasn't too bad. You're left with the raw hole, which could be covered with a patch. You could turn the negative into a positive and make that a corner reinforcement. I needed access to the end of the hem of a drawstring, but in your case, the grommets really don't weigh much--- the four I removed weigh 3.1g *total*. For the damage done and the weight of the patches, it doesn't seem worth the bother.