Well, I might confuse you more, but here goes:
Even though a hole is small, 3/16" or 1/4", a boring bar or reamer should get you a straight and aligned hole moreso than a drill bit. That is because a boring bar or reamer is generally stiffer than a drill bit. Drill bits wander, you can often see it as you are drilling a hole. And smaller diameter drill bits wander more than bigger bits.
Now, I did say "should" because it also depends on your ability to align your reamer or boring bar accurately in your cross slide. And, I would agree that a nice carbide drill bit will be really stiff and should give you a nice straight hole But carbide bits are in a class of their own - much different than your more common HSS drill bits.
And I agree with Joey, that a reamer/boring bar is used to true up and clean up a pre-drilled hole of nearly the same diameter.
None of the above matters so awfully much on a ferrule, other than you'd like a pretty good fit so that you get a good glue bond. But when you get around to installing your first joint pin, then alignment and straightness become paramount.
Finally, unless your reamer or boring bar are mis-marked, then they should give you a hole the size that they are marked, given that they are installed/aligned correctly. Do you have a caliper? You should, either digital or analog, doesn't matter so much as a reputable brand. Then you can use your caliper to measure your cutting tool. You might be surprised at how far off some drill bits can be, for example.
I can only imagine that your reamer/boring bar was not aligned exactly on-center with your headstock chuck centerline. That could mean the cross slide was out just a few thousands or the reamer/boring bar was at an angle to the centerline of the chuck.
Hope this helps.
Gary
P.S. I agree you've got to try things first - experience is the best teacher. When you don't get the result you expect, then come here, explain what you did, what your result was, and ask "What happened?" But yes, be careful.
P.P.S (or is it P.S.S?) Put that ivory ferrule away in a safe place until you get your procedure down pat. An elephant was killed so that you could have that little piece of ivory - don't waste it.
Thanks Gary, a wealth of info there, and no worries, I won't be drilling into that ivory ferrule until I get this completely figured out, I had the ferrule way before I got the lathe so I can hold onto it awhile longer.