The balls will never be as new as when they come out of the box.
I have a set of old Centennials and the numgbers and ring have "popped" or raised above the rest of the ball.
When you polish them you are using an abrasive and that wears them down.
If you strike those balls with your cue tip you are wearing them and scratching the surface.
As the balls slide across the cloth they get worn.
The chalk on the CB from the tip may transfer to the OB and cause some marks.
I think that some also become distorted out of round by hard impacts. But really not enough to cause any significance or that noticable. There was a article one time about the earth being more of an egg shape instead of a perfect sphere but if a CB were expand to the size of the earth in its proportions the earth would be more of a perfect sphere than that CB. The CB would also have deeper canyons and mountains so high that the earths highest mountain would be a mere hill.
If you look at a bowling ball the runs down an oiled lane there eventually become a track that is worn into the ball.
It is nice to play with equipment that is in perfect condition but in reality how and where will this ever be done except on your own table if you maintain it that way.