My experience so far has been similar. One of my two bottles, the smaller one in my case, started leaking a bit around the rim where the nozzle is. So far, that's been the only problem with the filter.
I've forced the water through the filter every time also, otherwise it takes quite a while. I think it would work by gravity alone, but pretty slowly. Maybe if you hang it high enough to have a larger hydrostatic head it would work reasonably well, but I haven't tried that.
I also found that the push-pull filter makes a handy faucet for washing one's hands; I place the bag full of water under one arm, tip it downward with the valve open, and squeeze gently with my arm. Quite convenient. :)