Are Fargo Ratings inflating?

[skip100]

Any Bradley-Terry model needs an identification mechanism - in other words, a peg needs to be placed to anchor the ratings.

The population of Fargo players is getting increasingly bottom-heavy as league and casual players are added by the thousands. If the peg (for instance setting the average player rating to 500) doesn’t change, then we will get inflation.

Clearly inflation is happening when this text remains on the home page: “800: A top world-class player. Fewer than 20 players worldwide have ratings that exceed 800.” As of today, 52 players have ratings above 800.

 

[mikepage]

The population of Fargo players is getting increasingly bottom-heavy as league and casual players are added by the thousands. If the peg (for instance setting the average player rating to 500) doesn’t change, then we will get inflation.

That would be a poor way to do the peg for that reason.

Clearly inflation is happening when this text remains on the home page: “800: A top world-class player. Fewer than 20 players worldwide have ratings that exceed 800.” As of today, 52 players have ratings above 800.

If we look at every player who two years ago had an established rating between 600 and 700 and has played at all in the last two years, we get about 5,000 players. Those are the good regional players.
The average robustness of that group was 1100 then and is 1700 now.
The average rating stayed the same within a tenth of a point at 635.8
52% went up in rating, and 48% went down.

If we allowed the AZB Tail Waggers to fix the ratings by fixing the number of players over 800 or defining a particular rating as the 50th best player or whatever, we would have to acknowledge we are claiming these 5,000 serious regional players have notably deteriorated in skill over the last two years.

The top players are outperforming those 5,000 players in the 600s by a bigger amount now than they were a few years ago. A more rational explanation is the top players are performing better on an absolute scale.

 

[skip100]

So many straw men in that post it’s hard to know where to begin.

You have to pick your poison here. You can accept the argument above based on a mysterious group of unnamed regional players, which means ignoring your lying eyes about the skill level of the top players changing dramatically over a 5-year period. Or you can just be realistic that the meaning of a particular rating will evolve over time based on changing populations and changing skill levels within those populations. And it doesn’t really matter anyway since the value is in the relative differences between players.

 

[ironmaiden]

Fargo ratings are garbage. In the past 5 weeks I was 8-2 and 7-3 x 4 and my fargo went up maybe 8 points. At least once every week I played and beat players that were 50 to 80 pts higher.

Last week I played on Wednesday. Was super tired and weak. Ended up with strep throat and missed Thursday and Friday work. Went 3-7. Rating dropped 5 pts.

It shouldn't be so easy to drop points. There are a lot of "league" players that lose during the week and magically win during tournaments.

We need a rating system for tournament only play. I also think the cost of the buy in should be a multiplier of the rating. If someone is winning big money tournaments then the points should go up faster than someone playing in a local 15 dollar Friday night tourney.

 

[ironmaiden]

So many straw men in that post it’s hard to know where to begin.

You have to pick your poison here. You can accept the argument above based on a mysterious group of unnamed regional players, which means ignoring your lying eyes about the skill level of the top players changing dramatically over a 5-year period. Or you can just be realistic that the meaning of a particular rating will evolve over time based on changing populations and changing skill levels within those populations. And it doesn’t really matter anyway since the value is in the relative differences between players.

Let me play devil's advocate. Have the player's skills increased or the accessibility of matches via Youtube and the internet made it appear that way.

I don't think the players on average are any better than before. It's the internet making players from all over the world easily accessible for viewing that makes it seem that way.

Watch some videos of an elderly Luther Lassiter and tell me this guy wouldn't tar and feather today's top players if he were in his prime.

He's one of many. Back in those days people weren't technologically distracted. They just played pool all the time.

I wouldn't be suprised if the top 3 players in the world today might not be in the top 3 if they could go back in time and play against the greats.

 

[garczar]

Let me play devil's advocate. Have the player's skills increased or the accessibility of matches via Youtube and the internet made it appear that way.

I don't think the players on average are any better than before. It's the internet making players from all over the world easily accessible for viewing that makes it seem that way.

Watch some videos of an elderly Luther Lassiter and tell me this guy wouldn't tar and feather today's top players if he were in his prime.

He's one of many. Back in those days people weren't technologically distracted. They just played pool all the time.

I wouldn't be suprised if the top 3 players in the world today might not be in the top 3 if they could go back in time and play against the greats.

Internet exposure has nothing to do with Fargo rates changes. Games won/lost and who you played(and their record too) is all that matters.

 

[ironmaiden]

Internet exposure has nothing to do with Fargo rates changes. Games won/lost and who you played(and their record too) is all that matters.

I didn't mention Fargo rating in that comment. He was implying that players are better than they used to be. I don't think so.

 

[garczar]

I didn't mention Fargo rating in that comment. He was implying that players are better than they used to be. I don't think so.

The whole thread pertains to FR fluctuations thus my post. I don't think players get better over time but some are FAR better at maintaining their speed over time. As a whole pro players are way better than in the past. The level of play and the depth's of fields is brutal compared to when i first started(late 70's/early 80's) watching top players.

 

[BasementDweller]

Here's another little theory...

The 2:1 win expectation just breaks down at some point. Mike would be able to easily dismiss this idea based on the data. The idea is if you take a 800 vs a 700 and they are playing Matchroom's version of 9 ball, I could envision a scenario where the 700 just can't keep up with the 2:1 ratio. Once the 800 level player has the break dialed in and the game gets more technical -- he just runs away with the match. It seems like that's what the top level guys are now doing with this format. Guess around the 820 level. Sort of like what SVB has done historically with template rack 10 Ball. As the top-tier guys get closer to MASTERY of a game, the rating differences grow. Ultimately, the game gets more binary -- either you can play it absolutely perfectly or you can't. If you can play at this level, you're going to live outside the 2:1 ratio. Maybe this could be part of what's causing the "inflation" at the top.

There's really just so much going on with Fargo Ratings that those of us that don't have access to the data can't understand, argue, or defend. Thinking about the 2:1 ratio, does anybody think a 725 could keep up against Dennis Orcullo playing one-pocket, while maintaining that 2:1 win rate?

I have no problem with one-pocket being used but it's possible this game could most dramatically showcase how the 2:1 win ratio could break down at the top end.

Just the random tonights of a AZ Tail Wagger.

 

[Ģüśţāṿ]

The top players are outperforming those 5,000 players in the 600s by a bigger amount now than they were a few years ago. A more rational explanation is the top players are performing better on an absolute scale.

How do you justify this comment?

A 600 player is a 600 player and same goes for an 800+. The top players aren't necessarily getting better, although there are more top players now in the Fargo system than there used to be.

The biggest change I can think of is that the top players are only playing other players of their own caliber more often than not, especially with Matchroom trying to limit lesser skilled players from entering the same event.

So saying that they are "outperforming those 5000 players by a bigger amount" seems a bit misleading. It's more likely there are less data points now for an 800 versus a 600, and many more data points for 800 vs 800 and 600 vs 600 respectively.

Since the top players are predominantly playing other top players almost exclusively, It would make sense that that the top players are slowly inflating.

A specific and measurable point of reference for what defines a top player would be a great anchor point, but even thats pretty subjective IMO.

 

[derangedhermit]

There's really just so much going on with Fargo Ratings that those of us that don't have access to the data can't understand, argue, or defend.

This is the basic issue, for at least a few here. We can read the history and evolution of Elo-type rating systems (especially as applied to chess), and wonder what choices FargoRate uses, and why those choices were made. Without access to the algorithms and data, it's hard to learn anything. Transparency also helps create trust.

I think FargoRate is the best rating system pool has ever had, and is very useful for comparing contemporary performance and for handicapping.
The OP has a long background in scientific research and academia, including publishing. FargoRate is approaching 300,000 players and 30,000,000 games. I would very much enjoy and appreciate a paper, or series of papers, with some rigor, on the application of the Fargo rating system to rating pool performance, and what has been learned about the rating system, its applications and limits, the sport, and players' performance.

 

[garczar]

Here's another little theory...

The 2:1 win expectation just breaks down at some point. Mike would be able to easily dismiss this idea based on the data. The idea is if you take a 800 vs a 700 and they are playing Matchroom's version of 9 ball, I could envision a scenario where the 700 just can't keep up with the 2:1 ratio. Once the 800 level player has the break dialed in and the game gets more technical -- he just runs away with the match. It seems like that's what the top level guys are now doing with this format. Guess around the 820 level. Sort of like what SVB has done historically with template rack 10 Ball. As the top-tier guys get closer to MASTERY of a game, the rating differences grow. Ultimately, the game gets more binary -- either you can play it absolutely perfectly or you can't. If you can play at this level, you're going to live outside the 2:1 ratio. Maybe this could be part of what's causing the "inflation" at the top.

There's really just so much going on with Fargo Ratings that those of us that don't have access to the data can't understand, argue, or defend. Thinking about the 2:1 ratio, does anybody think a 725 could keep up against Dennis Orcullo playing one-pocket, while maintaining that 2:1 win rate?

I have no problem with one-pocket being used but it's possible this game could most dramatically showcase how the 2:1 win ratio could break down at the top end.

Just the random tonights of a AZ Tail Wagger.

FR is not good at handicapping 1p matches. I know more than one 650ish 9b players that can/do smoke much higher FR players at 1p.

 

[Rocket354]

51 players on top 100 list are now over 800 (vs. 35 players two years ago)
The top rating is 848 (vs. 830 two years ago)

Are Fargo Ratings inflating? That's not an easy question to answer. But here is evidence they're not--or at least any drift is small. We compared the ratings now to the ratings of the same players two years ago including all players who had established rating then and also played at least some games in the last two years. Then we divided those players according to how many games they logged in the last two years.

Not surprisingly, the people who play more have ratings that increase more. The implication is that players are more likely to have improved if they play a lot. If there is a rating drift/inflation, we'd expect it to appear as an INTERCEPT on the vertical axis, a rating change not tied to improvement. The fact the intercept is near zero suggests there is at best little drift.

The more I think about this argument the less it convinces me.

This shows that players, no matter how many games they've played in the last two years, have on average increased their Fargo ratings.

If there was no drift I would expect to see an intercept on the x-axis. That is, players with few games played over the last two years actually getting worse on average.

To truly get something relevant I believe correcting for games prior to two years ago compared with games since would be a correct method. Those players with big increases showing improvement on average and those with big decreases showing decline would support the hypothesis that playing more leads to improvement (not that such a hypothesis really needs further support).

As it is, this analysis does show that people who play more have more of a change in their score and those who play less see less change--an already well-known feature of the system.

Thus, this analysis just reads as an attempt to twist statistics into selling the exact opposite conclusion that one would normally reach from the fact that on average every group of players no matter how many games they played saw their Fargo scores increase in the past two years. That is: score inflation.

 

[mikepage]

[...]

If there was no drift I would expect to see an intercept on the x-axis. That is, players with few games played over the last two years actually getting worse on average.

I think you are meaning their actual skill might deteriorate a bit if they're not playing. Perhaps. Keep in mind, though, that FargoRate wouldn't know about that and would need a LOT of games going into the system to track it. If people are not playing or are barely playing, that means FargoRate is basing its estimate of their new skill on their OLD games.

This estimate of their skill based on their OLD games would drift up if there was a general tide rise of their opponent ratings. It's like asking what happens to the rating of a player who dies? The answer is it stays about the same but is subject to whatever drift there is, inflation or whatever. Take, for example, Nelson Oliveira, good player from MA who played in Turning Stone and Joss tour, etc and died in 2018. He was 726 then and is 728 now. That few tenths of a point per year is the drift.

 

[mikepage]

You have to pick your poison here. You can accept the argument above based on a mysterious group of unnamed regional players, which means ignoring your lying eyes about the skill level of the top players changing dramatically over a 5-year period.

When you say mysterious group, it was EVERY player between 600 and 700 two years ago. When you say unnamed, there were 5,000 of them!
Here is every US player, named, who had an established rating over 700 2 years ago and logged at least some play last two years. Those in red (123 of them) went down in rating. Those in green (135 of them) went up in rating. Average of this group went down 2 points over 2 years, from 724.7 to 722.7.

 

[Rocket354]

I think you are meaning their actual skill might deteriorate a bit if they're not playing. Perhaps. Keep in mind, though, that FargoRate wouldn't know about that and would need a LOT of games going into the system to track it. If people are not playing or are barely playing, that means FargoRate is basing its estimate of their new skill on their OLD games.

This estimate of their skill based on their OLD games would drift up if there was a general tide rise of their opponent ratings. It's like asking what happens to the rating of a player who dies? The answer is it stays about the same but is subject to whatever drift there is, inflation or whatever. Take, for example, Nelson Oliveira, good player from MA who played in Turning Stone and Joss tour, etc and died in 2018. He was 726 then and is 728 now. That few tenths of a point per year is the drift.

Your last couple sentences suggest there is score drift/inflation. And provides much better insight into it than the OP did.

However, drift is not necessarily limited to that amount. Nelson's relative score has not been much affected by the influx of new players into the system. At least I would hope not--if a player who legitimately improves over time means 6-year-old games (and therefore the player's opponents from then) are now viewed much differently would mean FargoRate is a suspect system.

But the new players force the relative nature of the system to push particularly top players who are continuing to play and be evaluated out further due to more needed granularity.

Without knowing the exact algorithm an exact assessment of the system's trends is not easy.

The example of Nelson Oliveira shows how much the drift has affected 6-year-old games which are presumably de-emphasized, not how much active players today might be affected.

 

[garczar]

When you say mysterious group, it was EVERY player between 600 and 700 two years ago. When you say unnamed, there were 5,000 of them!
Here is every US player, named, who had an established rating over 700 2 years ago and logged at least some play last two years. Those in red (123 of them) went down in rating. Those in green (135 of them) went up in rating. Average of this group went down 2 points over 2 years, from 724.7 to 722.7.
View attachment 749445

I just hate it when FACTS get in the way of a good story. Kinda hard to argue with this chart. So many that don't like FR go by what they feel/sense whereas MP goes by the #'s.

 

[BasementDweller]

I just hate it when FACTS get in the way of a good story. Kinda hard to argue with this chart. So many that don't like FR go by what they feel/sense whereas MP goes by the #'s.

Are you including yourself in this list since you questioned Fargo as well?

FR is not good at handicapping 1p matches. I know more than one 650ish 9b players that can/do smoke much higher FR players at 1p.

 

[nine_ball6970]

I just hate it when FACTS get in the way of a good story. Kinda hard to argue with this chart. So many that don't like FR go by what they feel/sense whereas MP goes by the #'s.

What I find crazy is the number of players who dropped 30-70 points in a two year period.

 

[skip100]

I just hate it when FACTS get in the way of a good story. Kinda hard to argue with this chart. So many that don't like FR go by what they feel/sense whereas MP goes by the #'s.

With respect, that chart doesn't really show anything. It is a slice of people, all of whom are much more skilled than the typical Fargo-rated player, all of whom live in only one country, some of whom got better at pool, some of whom got worse at pool, some of whom played a lot, some of whom played barely at all, and some of whom had ratings go up and some of whom had ratings go down. It is not representative of any general deflation or inflation which may or may not be happening across the system as a whole.