My wife and I take our kids several times a year out there. Quite often we do the same (leaving Friday after work/school). We like to camp at the little pond near Willson Peak. You have to climb over 1600 ft in less than 3 miles to get there. We don't mind the climb, since we are conditioning for the JMT and like that the pond is off the trail and you are alone. In the morning we go then down to Kelly Lake for some fishing.
Willson Camp is less climb and more gradual, so you might prefer that (and the fact that you have a rest room). We have camped there too (one night in the rain under the little barn), but prefer Willson Peak.
I'm not sure what you mean by Fish Pond? I'm aware of Fish & Game Pond, but that is almost all the way at Coit Lake and way beyond your 4 mile radius. May be you mean Rodeo Pond (0.8 miles up the road from Willson Camp)? We have also stayed at Wasno Pond which was nice (and we saw a scorpion there), but that is roughly 5 miles. Wasno and Rodeo Ponds both offer rest rooms, if that makes a difference for you.
For Henry Coe we leave our SteriPen at home and use our gravity filter instead. The kids like the water better, when it is filtered and doesn't have the algae taste. You should be fine with a bandana as pre-filter and the SteriPen for treatment, but you might find a certain "taste" to the water. So you might want to bring some Gatorade powder, etc.
Have fun out there,