You are right, this is totally personal. I was a traveler before I took up hiking -- and I used to use different stuff sacks for different things (bag, pad, clothing, misc., etc.).
After a few trials, I found that I could pack more and faster and easier and lighter by ridding the sleeping bag, pad, and clothing stuff sacks! Now, I actually prefer just one big hole -- and use one small stuff sack to house the misc. small items.
When traveling, I prefer compartments for organization and ease of use. But as an UL hiker, what I find is that at the end of the day, pretty much all my stuff is pulled out anyway -- so no need for compartmentalizing at all. During the day, I do separate out my hat, rain jacket, first aid kit and food -- placing them in outer pockets and hip belt pockets. I LOVE hip belt pockets.
YMMV, of course.