I picked up a pair of a shorts almost exactly as you're describing, even in green, at my local thrift store. They were soccer shorts, complete with emblem. They weigh 3.9 oz. I think they were $3. I don't do liner (edit: lined) shorts either.
So thrift it up.
You might also find a lightweight pair of board-shorts, something with a shorter cut, they don't have liners.