My personal thoughts are- if they are really porous or weren't sealed well to begin with, then It's sometimes near impossible to get some of them completely clean with no specs. Most of the time I am able to, but occasionally I get one that seems as if nothing short of sanding the ferrule down would help, which really only leaves 2 options, get as clean as possible and live with It, or change the ferrule if the looks are really that important.
Again that's just My opinion, maybe others have better results, but I have used just about every method mentioned on here and then some, and while I get most of them clean, there is that occasional stubborn situation.
When I was a teenager I worked for a furniture store during a couple of summers delivering for them and helping out around the store between deliveries, and We use to use Softscrub cleaner on some of the PVC patio furniture to get the stubborn stains out that would get down into any scratches in the pvc. I have used It on some stubborn ferrules with a ME and It helps, but You have to use It sparingly or It can make a mess, and as someone mentioned You don't want to get It on the tip. I usually only use It as one of the last resorts, and when I'm going to install a new tip anyway, so that I can do a fresh facing after use, and cleanup. I don't recommend leaving the stuff on there, as It really needs to be cleaned off very well before finishing the ferrule up and tipping, but It's another option to add to the list of many methods.
With the magic erasers I don't know if anyone else does this, but I cut about 4 pieces out of one full size sponge, and instead of cutting them clean, I score them with My razor knife then rip a piece off. I then use that ripped, ruff, side to clean with. Not sure If It's just My imagination, but the ruff cut seems to reach down into the pores better then a clean cut does. I notice this more when cleaning shafts then the ferrules, but I still use the same method either way.