When I don't expect to get my pack wet, I pack my goods in a decently durable plastic "trash compactor bag" type pack liner, like the ones sold here or over at Gossamergear.com.
Last year, I had an unexpected swim of the upper Yellowstone River, so I ended up putting the pack liner on the outside and using some shockcord to close it in twist-tie fashion.
By the time I got to the other side, my goods were dry.
However, the pack was a huge pain in the butt to swim with. Not having a shoulder harness to loop over one shoulder while you're swimming would make fjord swimming laborious and makes river swimming difficult and risky.
I'd much prefer to have the pack harness available to me for swimming.