I'd say first you need to define sport:
Cambridge Dictionary:
"a game, competition, or activity needing physical effort and skill that is played or done according to rules, for enjoyment and/or as a job"
Merriam Webster:
"a physical activity engaged in for pleasure"
Collins:
"Sports are games such as football and basketball and other competitive leisure activities which need physical effort and skill."
All mention physical effort or activity, there is undeniably some physical activity in pool and the physical activity does effect your success. I suppose what decides whether it's a sport or a game depends on your threshold for how much activity is involved to cross the line from it being a game to a sport.
But if we can have motor sports as an accepted term I don't think it's unreasonable to have cue sports accepted too.