Greg said it well.
If I can take non-native fish, I will. Even more, if I can keep non-native invasive species, I would prefer that--I refuse, for example, to throw lake trout back alive in most of the watersheds I fish (not that I catch them, but it is a philosophical commitment). Even if I don't eat them, the lake trout do not get thrown back alive.
That said, I'm happy keeping a few medium-sized native fish if that is allowed by the fishing regulations and the fishery is vibrant and well-sustained. For example, on a recent trip to the South Fork of the Flathead, we had trout every night for dinner, but we never took our full daily or possession limit. Except for the first night, we ate native cutthroat. I feel fine with that decision--especially in the context of a backpacking trip, not as the norm every time I go out.