I have a permit for this Summer to enter the JMT at North Lake and exit the Whitney Portal.

I'm considering going the other way instead, North Lake to Tuolumne Meadows.

Do the authorities care about the change in direction? Or is the permit mostly about where you hop on?

The permit is definitely issued for your entry into the park/wilderness.

Your permit would not allow (legal) access for entering The Whitney Zone from the Portal. I suppose if you happened to make it past Whitney, you could probably talk your way through a curious ranger's questions, but around Whitney they are pretty strict. We were asked a few times if we had a permit last year on the JMT, but never had to actually produce it.

Dave, I am sure you can change your exit plans pretty easily. I would call the ranger station and see