It worked.
I just bought some super glue from the store, put some on the retriever stick (Underside of the wooden hilt), dropped it in the slot and pushed it through the hole of the spacer, waited about a minute and pulled it out with a twisting motion. Only bad part is that you can't use that...