How many matches are really needed to determine final rankings?

[tctc]

Originally Posted by Ether

I wonder how different the analysis would look if you looked at partial OPRs instead of partial average scores.

I put together partial OPRs for each of the 8 events you listed above, in case someone is interested enough to plot them or otherwise analyze/summarize them.

...

Column A is team number, Column B is final OPR (after all Qual matches), Column C is partial OPR after all Qual matches less one, etc

I created a couple plots based on the OPR data you provided. It looks like each column represents a new match and not a round. For example, Silicon Valley has 95 matches but 10 rounds. If that is the case, the data has only the last 20 matches. It would be easier to compare if the OPR was computed at the end of each round. But, the OPR graphs are still a little interesting. First, let's look at North Star.



The OPR for the final 20 matches seems to be fairly constant for teams. It looks like whenever a team has a match, there's a jump. And then in between matches, it might go back towards the mean. Beyond the top 10 teams, it gets very cluttered. That is, there aren't many OPR points distinguishing teams. So, for the Silicon Valley Regional, I took out all but the top 10 teams.



This makes it a little easier to see how the team's OPR changes between matches.

I didn't dive too much into correlating the OPR with the ranking because of the different domains. But, it appears that the OPR is better able to account for the top seed effect, as I like to call it.

Speaking of which, I wanted to revise my analysis from a couple posts ago, where I looked at how the rankings would change if the top team wasn't at the competition. The three categories are not mutually exclusive. The With status could include teams that also have matches against the top seed. Likewise, the Against status could include teams that also have matches with the top seed. So, I filtered the results a little differently to look at Only With, With And Against, Only Against, and Neither. See below for the North Star and Silicon Valley Regionals.



I also computed a few statistics using the student t-test. For both the NS and SV Regionals, if a team was either against or not at all in a match with the top seed, that team ended up having a lower rank (p<0.001). On the flip side, if a team was with the top seed or with and against it, then it did have a higher rank (p=0.007 for NS With And Against, p<0.001 for all other cases).

So, the conclusion is the same - a team does better if it's with the top seed and worse if it's against or not with the top seed - but I think this method proves the point better.

Cheerio.