Steven, I enjoy these types of discussions because every mountaineer has an opinion, yet no one is truly right or wrong. Endless arguments over beers at the lodge..
Take my words with a grain of salt because I've been studying mountaineering for exactly two winters, mainly by the books and balls method; no one-on-one instruction..
Anyway, my short answer is 50cm is fine for what I think you describe; piolet ancre and piloet asis on short class 4 or 5 where loosing your balance means a short bumpy ride for just a few meters. I assume you are not roped to anyone, so your short axe used in self-belay could reduce the chance of a slide, or in self-arrest reduce the length of that slide. I've tried short axes in belay and arrest positions, and they work fine. Where a long axe is more useful is in piolet canne, and you said you don't do that.
I'm old school in my choices, and tend to go with what Chouinard says in "Climbing Ice"* Paraphrasing, 'an axe becomes essential when it becomes useful". It becomes useful depending on what you want to do; if I need to do a boot belay, 55 or shorter is fine; chop steps 70, screw in a piton 50, cows tail anchor 50, piolet canne 75 or 80!, etc..
As Choinard says, a 70cm is a good compromise when carrying only one tool.
More important than length of the tool is the motion in the ocean.. oh wait, crossing my metaphors.. I mean knowing how to use it. Choosing a classically shaped steel head with the biner hole directly over the shaft, neutral tip, no leash rings, etc..
I have a BD 65 for when I expect to need an axe, a Camp XLA 210 when I don't (and a 1972 bamboo handled Chouinard 60 when I want to look like a crusty mountaineer who advises people on internet forums)
If you haven't read Climbing Ice; I highly recommend it; along with Mountaineering Freedom of the Hills, and the Mountaineering Handbook. (forgive me if I am lecturing to someone more experienced than me; maybe the lurking newbies will take my advice :)
*ISBN 0 340 23283 8