> Sharpie in LARGE letters on one of my Tyvek groundsheets
+1 to one Dave T from another.
If you don't already have a large, white, waterproof, tearproof surface along, a Fedex shipping envelope can be inverted and a message written on each of the two large surfaces. If you feel guilty about scoring one from the Fedex supply dispenser, leave a quarter on the counter.
Yes, use LARGE letters.
Such an envelope could also be multi-purposed as a stuff sack of sorts, a waterproof place to keep maps and permits, a rain hat in a pinch, and you could always mail stuff home with it.
Your backpack and location may already communicate "Hiker to Town". Other important techniques:
Only use a sign indicating a short trip, not the long one you're hoping for. Strike up that conversation after the first few minutes after they've realized their head is still attached and therefore you're not an ax murderer.
Clean yourself up as much as possible. Remove the larger sticks and rocks from your hair and beard.
Display the most interesting/foreign flag you have any remote right to display.
And, of course, smile.