If you are in the lower 48 states, you should worry about:
bee stings (if you are allergic)
falling off rocks
falling into swift (or not so swift) water
Then, there is the remote chance of bear attack (if you're in grizzly territory), snake bite (depends on which snake and where whether you should be paying attention to the ground or the trees or the water - in California the only venomous snakes are rattlesnakes, and they don't chase you down to kill you, so if you keep an eye out and not bother them you should be fine), deer (specially in rutting season), or of a black bear maybe taking off with your food (which is why many places require you to carry a canister or hang your food).
This would be going by statistics - deer do kill more people than bears by far.
You can access an article on animal related deaths here - http://scark.org/docs/Animal%20Related%20Fatalities.pdf
Note that domesticated animals kill more people than wild ones.
Most bugs that are venomous here in the states are not deadly - they'll cause illness, perhaps. The statistics are pretty clear on the level of threat scorpions, centipedes, black widows, etc represent. (I don't even worry about them at all.)
Ticks, which are not venomous, carry disease that can be deadly, so should be dealt with - DEET or clothing (treated or not), and daily tick checks to remove any sneaky ones that got through. Not hard to do.