Fargo Rating?

[mikepage]

The fargo system is in its infancy. We can learn from the history of chess since it's elo system has been around far longer.

After a couple minutes of googling, I found a source of this exact problem in chess. It's very similar to the situation in pool. Most women play almost exclusively in women events, while some play against men. They actually manually added 100 points to all female players because of this issue:

http://www.3-dbaseball.net/2015/05/gender-in-chess-part-2-elo-ratings.html

5th paragraph

This is not a very good comparison. Chess was doing goofy things at the time (before the adjustment) that actually treated women differently from men--like bringing men and women into big tournaments with a rating floor hundreds of points apart.

Pool are Fargo Ratings are in a much different situation.

I'm going to give a geography example before discussing the male/female rating comparison.

Suppose the entire world of pool players--with the exception of one isolated island in the south pacific--all have accurate ratings. Every player is rated accurately with lots of games and--off the island--we do a good job predicting the results of a matchup between any two players.

The island, though, has a group of 100 players none of whom has played off the island. But these 100 people play one another all the time and are rated accurately relative to one another.

The problem is the islanders, as a group, are 100 points too high.

A 600 on the island plays like a 500 in the rest of the world
An 800 on the island plays like a 700 in the rest of the world, and so forth


Now let's suppose SVB goes on a fishing trip with a friendly host someplace in a small boat and gets stranded on that island.

Shane plays the local hero, who has the same "rating" as Shane but really plays 100 points lower. They play a race to 100 and Shane wins 100 to 50. Then Shane gets picked up by a rescue boat

What happens?

It depends on whether the rating update algorithm is a sequential one (the easy way to implement elo-type schemes) or a simultaneously optimized one (what we call ab initio global optimization).

In the sequential approach, SVB and the local hero exchange a bunch of rating points because of Shanes big win. But the other islanders don't change yet. Then when the now-lower-rated hero plays an island shortstop, some of the hero's rating adjustment is rubbed off on the shortstop, and so forth. Eventually the whole island is down (in the right direction) but only by a little bit. It will take many more stranded boats of pool players to really fix the problem.

The ab initio global optimization--what we do--is very different. With our approach, as soon as Shane leaves the island, every player on the island is lowered by 100 points. Every single one. The only connection between an average player on the island and a player in Peoria IL is through the hero's match with Shane. So to the extent Shane or the hero had a good day or a bad day in that 150 games, the island could be shifted 20 points or so one way or the other. But they are in the ballpark.

If another boat with an average league player from Springfield OR gets stranded on the island, and that player plays another 150 games with an islander (or 15 each with 10 islanders), then there is 30 total games coupling the groups. Now things are pretty much in sync.

more on men/women in pool later

 

[railbird99]

This is not a very good comparison. Chess was doing goofy things at the time (before the adjustment) that actually treated women differently from men--like bringing men and women into big tournaments with a rating floor hundreds of points apart.

Pool are Fargo Ratings are in a much different situation.

Please elaborate on the "goofy" things. The rating floors didn't really affect top level chess, where the issue was prevalent. The issue was caused by the the same thing that's wrong with pool. It seems like a good comparison to me.

The ab initio global optimization--what we do--is very different. With our approach, as soon as Shane leaves the island, every player on the island is lowered by 100 points. Every single one. The only connection between an average player on the island and a player in Peoria IL is through the hero's match with Shane. So to the extent Shane or the hero had a good day or a bad day in that 150 games, the island could be shifted 20 points or so one way or the other. But they are in the ballpark.

I'd love to hear more about this approach. If Shane is stranded on the island and plays only one set, like a race to 9 with one person on the island, how can you justify adjusting everyone's rating points based on that one set? What if the islander gets lucky and wins, and you've just adjusted everyone's rating in the wrong direction? How do you determine when and how much to adjust all of these ratings?

 

[mikepage]

Please elaborate on the "goofy" things. The rating floors didn't really affect top level chess, where the issue was prevalent. The issue was caused by the the same thing that's wrong with pool. It seems like a good comparison to me.

I don't know much about the issue. I was reading this link.

http://anusha.com/schu-pol.htm

I'd love to hear more about this approach. If Shane is stranded on the island and plays only one set, like a race to 9 with one person on the island, how can you justify adjusting everyone's rating points based on that one set?

How can we not? That one coupling match is the ONLY link between the performance of a player on the island and a player elsewhere. Recognize that before the coupling match, either group, the island or the rest of the world, could have its ratings shifted up or down a thousand points and have nothing change. Right now if we raised everybody in the real system by 500 points or 50 points or whatever, nothing would change. It is only rating differences that matter.

So when you have two completely isolated groups, there is no correspondence between the ratings. You have to have coupling between the groups to have the numbers mean the same thing.

What is key, though, is it doesn't matter where that coupling occurs. It could be at the shortstop level, at the league level, or at the top level. Any of these serve to shift the two groups into alignment.

What if the islander gets lucky and wins, and you've just adjusted everyone's rating in the wrong direction? How do you determine when and how much to adjust all of these ratings?

By the principle of maximum likelihood

https://en.wikipedia.org/wiki/Maximum_likelihood

If the adjustment happened based on one race to 9, we would say these groups are very very poorly coupled. And yes we could be off by hundreds of points if that one coupling match was a statistically unlikely event. But if the two groups are coupled by thousands or tens of thousands of games, then the coupling will be good.

 

[railbird99]

How can we not? That one coupling match is the ONLY link between the performance of a player on the island and a player elsewhere. Recognize that before the coupling match, either group, the island or the rest of the world, could have its ratings shifted up or down a thousand points and have nothing change. Right now if we raised everybody in the real system by 500 points or 50 points or whatever, nothing would change. It is only rating differences that matter.

In the scenario where Shane plays one short set on the island then leaves, and the islander happens to beat him, you now have rating differences that are even more inaccurate than before there was any coupling.

It seems like you are trying to make a small insignificant result have more of an impact than it should.

But if the two groups are coupled by thousands or tens of thousands of games, then the coupling will be good.

If the two groups need to have a sufficient amount of coupling matches for this approach to have any kind of accuracy, then it seems like the approach.itself suffers from the same problem it was meant to solve.

 

[mikepage]

In the scenario where Shane plays one short set on the island then leaves, and the islander happens to beat him, you now have rating differences that are even more inaccurate than before there was any coupling.

But you are responding from an omniscient perch that you don't really have. I arbitrarily said one group is 100 points higher than the other. We actually have no information about how one group plays relative to the other UNTIL that coupling match happens. Then that coupling match contains all the information we have.

If the two groups need to have a sufficient amount of coupling matches for this approach to have any kind of accuracy, then it seems like the approach.itself suffers from the same problem it was meant to solve.

I am not following this.

 

[railbird99]

I am not following this.

You admitted that if the coupling were determined by one or a couple matches, that it could very likely be way off.

Then you said once there are hundreds or thousands of matches, there will be a good coupling.

The initial problem that this approach is meant to solve is the fact that there are not enough matches between these groups for the ratings to be comparable. But then you say in order to have a good coupling and for the ratings to be accurate, this approach requires a hundreds or thousands of games.

So how is this approach any better than the original ELO algorithm where every match doesn't affect every player in the isolated group? They both seem to require many matches between the groups.

 

[mikepage]

You admitted that if the coupling were determined by one or a couple matches, that it could very likely be way off.

Then you said once there are hundreds or thousands of matches, there will be a good coupling.

The initial problem that this approach is meant to solve is the fact that there are not enough matches between these groups for the ratings to be comparable. But then you say in order to have a good coupling and for the ratings to be accurate, this approach requires a hundreds or thousands of games.

So how is this approach any better than the original ELO algorithm where every match doesn't affect every player in the isolated group? They both seem to require many matches between the groups.

Ah, OK I see.

Our way 20 or 30 races to 9 (which is 350-450 games) between any members of either group solves the whole problem.

With the sequential approach that doesn't even barely makes a dent in the problem. If the groups are large, you may need 100 times this to start getting things in line.

 

[Eric.]

Ah, OK I see.

Our way 20 or 30 races to 9 (which is 350-450 games) between any members of either group solves the whole problem.

With the sequential approach that doesn't even barely makes a dent in the problem. If the groups are large, you may need 100 times this to start getting things in line.

Mike,

Without giving away your proprietary info, can you explain the similarities and differences of FargoRate vs Accu Stat's method?


Eric

 

[JC]

This is not a very good comparison. Chess was doing goofy things at the time (before the adjustment) that actually treated women differently from men--like bringing men and women into big tournaments with a rating floor hundreds of points apart.

Pool are Fargo Ratings are in a much different situation.

I'm going to give a geography example before discussing the male/female rating comparison.

Suppose the entire world of pool players--with the exception of one isolated island in the south pacific--all have accurate ratings. Every player is rated accurately with lots of games and--off the island--we do a good job predicting the results of a matchup between any two players.

The island, though, has a group of 100 players none of whom has played off the island. But these 100 people play one another all the time and are rated accurately relative to one another.

The problem is the islanders, as a group, are 100 points too high.

A 600 on the island plays like a 500 in the rest of the world
An 800 on the island plays like a 700 in the rest of the world, and so forth


Now let's suppose SVB goes on a fishing trip with a friendly host someplace in a small boat and gets stranded on that island.

Shane plays the local hero, who has the same "rating" as Shane but really plays 100 points lower. They play a race to 100 and Shane wins 100 to 50. Then Shane gets picked up by a rescue boat

What happens?

It depends on whether the rating update algorithm is a sequential one (the easy way to implement elo-type schemes) or a simultaneously optimized one (what we call ab initio global optimization).

In the sequential approach, SVB and the local hero exchange a bunch of rating points because of Shanes big win. But the other islanders don't change yet. Then when the now-lower-rated hero plays an island shortstop, some of the hero's rating adjustment is rubbed off on the shortstop, and so forth. Eventually the whole island is down (in the right direction) but only by a little bit. It will take many more stranded boats of pool players to really fix the problem.

The ab initio global optimization--what we do--is very different. With our approach, as soon as Shane leaves the island, every player on the island is lowered by 100 points. Every single one. The only connection between an average player on the island and a player in Peoria IL is through the hero's match with Shane. So to the extent Shane or the hero had a good day or a bad day in that 150 games, the island could be shifted 20 points or so one way or the other. But they are in the ballpark.

If another boat with an average league player from Springfield OR gets stranded on the island, and that player plays another 150 games with an islander (or 15 each with 10 islanders), then there is 30 total games coupling the groups. Now things are pretty much in sync.

more on men/women in pool later

It only took one stranded boat of Norway rats to end the bubonic plague

JC