I don't use a Spinn Twinn, but I do regularly use a Spinnshelter. It does require a good taut pitch to minimize deflection, but I don't find that true with "light" wind, more like gusty conditions. If light wind has got you down I think either you're not pitched taut enough, or your pitch is slightly off, which will make it impossible to tension the ridgeline properly. Spinnaker doesn't stretch, so it's not very forgiving.
In the photos above it looks like your rear pole may be just slightly too long, hard to tell, have you ruled that out?