Here's my wild a$% guess. I think it might be a combination of tradition and practicality.
Back in the old days like 1800's Carom games were the most popular games, at least much more popular than today in this country. Since Ivory was the preferred material there was a problem with the balls getting out of round and needed to be trued which made them smaller each time. If you start off with a large ball and need to take some off to make it round the percentage difference is less than if you start off with a small size. Thus a smaller effect in play. Over time and several cuts to make them true they became too small for Carom and were cut to Pool or Snooker balls which is the practical side of my argument. Reducing the expense of purchasing new ivory, just recycle and convert to Pool and Snooker.
Form the physics standpoint I'll probably agree with the comments above regarding kinetic energy/distance and possibly easier to apply exacting English with a slightly larger target area.