I've been a whitewater raft guide for 25 years, taught raft guides for over 15 years, and taught trainers for over 10 years, as a volunteer for Friends of the River.
I have been swept off my feet crossing a river (with a lifejacket, not a pack), and fallen out of boats into rivers. I've been rescued, rescued myself, and rescued other people.
I watched someone drown 100 yards away who was wearing a lifejacket, but we couldn't get there in time.
You take deadly risks crossing rivers, even with a lifejacket and training.
Do not think you will stay calm and rational when you get swept off your feet. Many people panic or freeze, even after some training. I did.
A pack that floats, strapped to your back, will try to float you face down in the river. Makes breathing more challenging.
Even if you don't panic, and you ditch your pack, and you are a strong swimmer, you can swim into more dangerous situations – strainers, holes, foot traps, and others – if you don't know any better.
We don't seem to have classes on “crossing rivers for backpackers” in USA.
Get swiftwater rescue training instead. One day classes start at $95. Look for courses aimed at recreational users (fishers, boaters) rather than professionals. You will learn how to cross rivers more safely, how to rescue yourself when things go wrong, and how to rescue others.
If you aren't going to take a class, at least read a good book. Standard disclaimer applies: Reading a book is NO SUBSTITUTE for training. Really and truly.
Good books include:
River Rescue: A Manual for Whitewater Safety, by Les Bechdel and Slim Ray.
Whitewater Rescue Manual: New Techniques for Canoeists, Kayakers, and Rafters, by Charles Walbridge and Wayne Sundmacher.
Stay safe out there.