I think the OP is thinking of charcoal as in BBQ charcoal or wood ash. The charcoal used in water filtration is little hard carbon rocks that don't disolve in water. The same as in aquarium filters and Britta filters. In fact I have used aquarium carbon inline with a filter.
It neutralizes chlorine and, I may be wrong, pulls some metals out of water.
You want to pass your water through the carbon as the final stage of filtration, not the beginning. The carbon is a filter stage, not an additive. You don't put the carbon in your water, you will pass your water through it.