No, its not out of hand but, with handicapping there are always going to be instances where the handicap is out of line. Your scenario above makes perfect sense but, when you try to apply that across the board and you have new players, out of state players, players who havent played in awhile, well......it gets quite complicated. As a player the best chance of me NOT being screwed by handicaps is to not have them.
Another note is that once you make handicapped tournament the norm, you can never have a successful open tourney. All the players that get spots in handicapped tournaments are not going to come out to these. They will get the mentality that if cant win with handicaps, how can they win.