You are probably tensing them. Like when you see a movie with a proper English woman drinking tea, and she keeps her pinky sticking out. After a few hours of tensing your pinkie I can imagine they would get sore.
Relax the pinkie. Let it just hang. So long as it isn’t getting in the way there is no reason to keep it tense, and for most people it is short enough compared to the other fingers that it won’t get in the way.