So I decided to try the sandpaper route earlier today and it did the trick, thank you for the tip!!! Next time I go to do this again after I go through 36 ferrules, I'll definately try the tap THEN part off lol
Thanks so much everyone for the advice!
Nathan