You were 100% right. I've never heard of a tourney that doesn't call that a foul since the cueball was in play.
You could have told him that you figured he wanted them there since he was the one who moved them.
This is my kind of smart assery and literally made me laugh out loud. Thank you, internet stranger, for brightening my day.