...it's about consistency
It isn't a foul if the referee didn't declare it.
Why don't you collect some fouls on video that weren't self-called and demand that the match be replayed? Persist in your demand and you will be considered mentally unstable by everyone including the opposing player. Some might write it off as just bad eyesight on your part in conjunction with mental instability.
Do you think everyone in the audience should be able to call fouls? What assurance do we have that all the fouls you think you've seen aren't just errors on your part?
So, if the rule is, the tip of your cue can't touch any ball other than the cue ball. And it does.
Did you commit a foul?
It does not matter if there is a referee present or not. Did you commit a foul?