Yes, there are drawbacks! If you don't have the fast twitch muscles needed, you risk doing all sorts of bad moves to get power and throw off your shots! The thing with a long backswing that some neglect to mention, is that it allows a slow, controlled start to the forward swing, which will help you keep the cue on line (as long as you stay still). When someone takes too short of a backswing (for them) they tend to tense up and jerk their stroke. What happens from that is that the cue will dip/rise or go left or right of intended contact point.Watch beginners (try to) hit big draws to get the idea, they either scoop or don't get draw at all. With a long backswing, the power will come without, effort, twisting or anything like that, just the length of the backswing and momentum building through the stroke. You can keep the intended cueball contact point, because you don't have to jerk the stroke or twist your body to get more force.
You can still have a short backswing on the short shots, but have power in reserve for the hard shots. And the good part is that the shots will feel the same, only the backswing will change. The accelleration etc will be exactly the same, only longer for the harder shots.
Watch this shot to see the beauty of it:
https://youtu.be/73Vd6SnP7A8?t=49