Yes, it is from the pocket. I can clean the balls spotless and when they go into the pocket they will have a contact oily mark on the ball from the pocket. I keep the pockets clean too.
Do your pockets have a cloudy residue on the surface?
I keep mine clean of this residue, maybe I need to...