Well, since you invited me, how can I say no?
There really isn't much that hasn't already been said by others in this thread. Back when I started, more than 20 years ago, there were significant discrepancies between APA areas. But over the years our knowledge, tools, practices, and training have all improved significantly, so it's way better now than it was then. This interweb thing has us all super-connected now too, so that helps.
Most differences today from area to area are due to one or more of three things. First, scorekeeping. The defense has been mentioned here and it's important. Those areas where it's drilled into the teams tend to have the most accurate skill levels. I tell my players "95% of the defensive shots you mark won't affect the player's skill level, but if you don't mark them 5% of the players you play will be under-rated."
Second is the vigilance of the league operator. We are all supposed to review the skill levels of all of our players weekly. Most of us do, and we all do it better than we used to, but with over 300 operators there will be some differences in the scrutiny. Any operator who tells you it's all software is either lying to you or not doing their job. There is no system in the world (NONE) that can be 100% correct without some subjectivity. Whether that's players who cheat or just can't be quantified mathematically for whatever reason, the review is critical and should be done locally, not by someone halfway across the country who doesn't know anyone in the local area. But again, getting everyone to do it exactly the same way is pretty much impossible too. Skippy was right, we can raise a skill level above what the software says it should be, but we can't lower it without getting approval from the national office.
Third is simple perception. Nearly every league operator hears from their members that players from other areas are rated lower than they are. The fact of the matter is that you know your own game and that of the people you play frequently way better than someone you rarely see, and the impressive parts of a stranger's game stick in your mind. I have been part of the Observer program at national events. Observers don't know the skill levels of the players they watch, and for the most part the observations come in right on the money, or maybe one higher or lower, depending on whether the player played a good match. Occasionally someone seems way off, and we do our best to find those people, but it's a very low percentage. However, like I tell my players, whenever you encounter those people at a national event, they are always from somewhere else, because pretty much everyone there is from somewhere else.
