As you so eloquently stated the rule above:
"If no one is called to observe the hit, the call goes to the shooter."
I've seen way too many teams and friends try to bully people into giving up ball in hand but they obviously don't know the rule. They should have gotten a 3rd party to watch the hit.
Since you asked, Here's my thoughts. It seems to me that from your description of what happened, you really hate that you lost, you don't like the player that beat you and you are making excuses to try to make yourself feel better.
Dropping out of the losers bracket and trying to make a scene probably had little effect on anyone besides you and reminds me of kids on the playground who whine saying "I'm not going to play with you anymore". That behavior is in my opinion childish and immature.
It's a game some you win and some you lose. If you get to shoot, and you loose it's most likely your fault for the loss. If you don't get to shoot, then it probably wasn't your fault.
I know this sounds harsh and I apologize if it offends you but you did ask for our opinions and I have given you mine.
It may be worth more or less than others depending on your perspective.
It may also not mean anything at all, after all it's just my opinion and I don't know you.
Of course it's quite possible that your opponent knows that he fouled and in that case, he should have owned up to it in my opinion.
However, the rules don't stipulate voting for who thinks it was a foul or not.
The call goes to the shooter if you don't have a neutral party watching the hit.
Hopefully, you'll do better next time.