Well, it's not exactly what you had in mind, but I spent 2 days doing 4 hour hikes on either side of Gaviota SP, 30 miles from SB. The camping in the park sucks, it's an asphalt lot designed for beach bums to put up tents from walmart and run generators, but it's surrounded by 2800 acres of beautiful foothills. If you're a plant geek, so much the better.
I've hiked out there a bunch and have never seen anyone else in the hills except at trailheads on weekends. I don't think there's any wilderness camping permitted there per se, but I don't know that it's prohibited either. It might be worth looking into, though beware the wildcats.
If you come across the sulfur springs, admire them from a distance, they are filled with chiggers.
Pics of the 2 day trip below: