Quote:
Originally Posted by JC
A 100 point spread between players predicts that the higher player will win two games for every one game the lower player wins.
But all games are not created equal and all 100 point spreads are not linear.
[...]

This is an interesting issue, and in principle this concern is legitimate. That is, there is nothing inherent that requires the ratings work the same way for different games.
In fact it is not hard to invent a game for which they clearly do not. Here is an example. I'm going to invent a game called SIETE. A single game of SIETE consists of a race to 7 9ball. The first one to 7 in 9Ball wins a single game of SIETE. The 100pointlower player using current Fargo Ratings will not win a third of the games of SIETE as expected but rather will hardly ever win a game. So mixing 9Ball data and SIETE data would be apples and antelopes.
So what then is the justification for mixing 8Ball and 9Ball?
Fargo Ratings started out described as a 9Ball rating system (2002 Billiards Digest article). Then by 2010 a simplified version of it was being used as an 8Ball rating system, with 13,000 games amongst 320 players. Over the next several years we started collecting some 9Ball data but didn't consider combining the data for two reasons: (1) we bought into the conventional wisdom that people's 8Ball and 9Ball speeds were two completely different things, and (2) the kinds of things that John mentions here.
By the fall of 2014, we had 20 times as much data and 20 times as many players, and we were doing separate ratings for 9ball (and 10Ball) and for 8Ball. And in fact we were doing separate ratings for different table sizes. We started noticing that the players largely had the same ordering and largely had the same rating gaps. So we started investigating. Did 8ball and 9Ball work the same? Did 7foot and 9foot ratios work the same? We had enough data to determine that remarkably so they did.
There is a huge incentive to combine data if it is at all reasonable to do so because many of your opponents who would be unestablished on, say, 9Ball on 7foot tables, are established when we consider other games. Think about this. Suppose you are offered a 7foot 9Ball big gambling match against some unknown opponent named Vilmos, and you ask a friend how Vilmos plays. Your friend says he has never seen Vilmos play on a 7foot table, but on a 9foot table he plays even with Oscar Dominguez. Are you going to think this is irrelevant knowledge? Of course not. Combining games and table sizes allows the FargoRate optimization to take advantage of similar situations, and these situations are the rule rather than the exception.
JC mentioned that I often use data from pro players in my analytical examples. There are two reasons for this. First we have a lot of data on top players. And second, people are familiar with the names and have seen them play.
What I've done here is to look at my own data. I am rated 623. So I think to investigate John's point, I should compare myself to someone rated 523 in both 8Ball and 9Ball. It is hard to get enough numbers to be statistically meaningful. But here is what I did. I looked at my own record against opponents within 20 points of 523 (from 503 to 543) in both 8Ball and 9Ball. This should average about a 100 point gap. At 100 points, I am expected to win 67% (two thirds) of the games.
I played 366 games of 9Ball against opponents 503 to 543 and 981 games of 8Ball.
I won 66.9% of the 9Ball games
I won 68.0% of the 8Ball games
It works out about as expected.
Now, with 6.4 million games in the system, we are up another factor of 20 from the Fall of 2014. We have plenty of data to analyze all sorts of things. We are not wedded to any particular thing we are doing. If the data suggests there is a better way, there is no reason we wouldn't just switch to that way.
Taking deep dives into our data is part of the fun for numerophiles like me ;)