How many matches are really needed to determine final rankings?

[tctc]

Originally Posted by Ether

I wonder how different the analysis would look if you looked at partial OPRs instead of partial average scores.

I put together partial OPRs for each of the 8 events you listed above, in case someone is interested enough to plot them or otherwise analyze/summarize them.

...

Column A is team number, Column B is final OPR (after all Qual matches), Column C is partial OPR after all Qual matches less one, etc

I created a couple plots based on the OPR data you provided. It looks like each column represents a new match and not a round. For example, Silicon Valley has 95 matches but 10 rounds. If that is the case, the data has only the last 20 matches. It would be easier to compare if the OPR was computed at the end of each round. But, the OPR graphs are still a little interesting. First, let's look at North Star.



The OPR for the final 20 matches seems to be fairly constant for teams. It looks like whenever a team has a match, there's a jump. And then in between matches, it might go back towards the mean. Beyond the top 10 teams, it gets very cluttered. That is, there aren't many OPR points distinguishing teams. So, for the Silicon Valley Regional, I took out all but the top 10 teams.



This makes it a little easier to see how the team's OPR changes between matches.

I didn't dive too much into correlating the OPR with the ranking because of the different domains. But, it appears that the OPR is better able to account for the top seed effect, as I like to call it.

Speaking of which, I wanted to revise my analysis from a couple posts ago, where I looked at how the rankings would change if the top team wasn't at the competition. The three categories are not mutually exclusive. The With status could include teams that also have matches against the top seed. Likewise, the Against status could include teams that also have matches with the top seed. So, I filtered the results a little differently to look at Only With, With And Against, Only Against, and Neither. See below for the North Star and Silicon Valley Regionals.



I also computed a few statistics using the student t-test. For both the NS and SV Regionals, if a team was either against or not at all in a match with the top seed, that team ended up having a lower rank (p<0.001). On the flip side, if a team was with the top seed or with and against it, then it did have a higher rank (p=0.007 for NS With And Against, p<0.001 for all other cases).

So, the conclusion is the same - a team does better if it's with the top seed and worse if it's against or not with the top seed - but I think this method proves the point better.

Cheerio.

[Ether]

Originally Posted by tctc

Silicon Valley has 95 matches but 10 rounds. If that is the case, the data has only the last 20 matches.

No, the data uses all 95 matches, then the first 94 matches, then the first 93 matches, .... etc:

Column A is team number, Column B is final OPR (after all Qual matches), Column C is partial OPR after all Qual matches less one, etc

Column B is the OPR using all 95 matches
Column C is the OPR using the first 94 matches
Column D is the OPR using the first 93 matches
.
.
.
Column U is the OPR using the first 76 matches


... so the progression from Column U to Column B shows how the OPR changed over the course of the last 20 matches.



Partial OPRs for all 106 events in Weeks 1 through 6 are posted here:

http://www.chiefdelphi.com/media/papers/3125

[tctc]

Originally Posted by Ether

... so the progression from Column U to Column B shows how the OPR changed over the course of the last 20 matches.

Okay, I see. Thank you for clarifying. Is it at all possible to get the OPR for all matches, not just the last 20? I understand that the algorithm needs every team to compete at least once before the OPR can be computed, which means I don't expect the OPR for the first 10 or so matches. But, ya, it could be interesting to see a bigger picture of how the OPR changes over the course of the competition. Cheers.

[Ether]

Originally Posted by tctc

Is it at all possible to get the OPR for all matches, not just the last 20?

Attached is a ZIP file containing partial OPRs for all 109 events in Weeks 1 through 7. Would you like fries with that?

Just kidding.

Let M be the number of qual matches at an event, and T be the number of teams.

The analysis proceeds as follows for each event:

for (k=M; k>T/2;k--){computeOPR(); deleteMostRecentMatch();}


PS - forgot to mention: I can transpose the rows and columns if that would make it easier for you to do your plotting.

[tctc]

Oooooohhhh.... preettttyyyy....

Had time for just a couple plots. But these really show what I was expecting: lots of movement/changing early on and then a leveling-out. However, the leveling-out really isn't that level. It seems like there are still jumps after teams have matches later in qualifications. Perhaps because of good alliance partners, or perhaps the team had a good match.

The first chart has all teams plotted from the Silicon Valley Regional, and the second plot has just the top ten OPR teams from the North Star Regional.



Thank you for all the number crunching to get all the OPR scores, Ether. Perhaps later there will be time for more analysis with reference to rankings instead of just graphical. Or, has this analysis gone on long enough? What else would be interesting?

[Alex2614]

Originally Posted by tctc

What else would be interesting?

I'd like to see an analysis of how much different things would be if the old W-L-T structure was still there. Because I know that my team would not have been in the top 10 at our regionals if the old structure was still in place.

Some say this structure is harder to win because you have to out score every team, not just your opponents. But I think it really does ensure that the best teams come out on top, if at the expense of a less exciting rank-watching time during the events.

[tctc]

Originally Posted by Alex2614

I'd like to see an analysis of how much different things would be if the old W-L-T structure was still there. Because I know that my team would not have been in the top 10 at our regionals if the old structure was still in place.

Which regional were you at? Also, what should be used as the 2nd order tie-breaker? Since average score is handy, I think I would prefer to use that. If not that, are coopertition and auto points available for each match? Do you know where one would go to find them?

Originally Posted by Ether

The actual elapsed time to get from raw data to those reports for all 109 events (6398 partial OPR reports) was only 17 seconds on a single core of a Pentium D in an 8-year-old machine running XP Pro SP3, using AWK to wrangle the data and Octave to crunch the linear algebra.

Nice!

[Ether]

Originally Posted by tctc

Thank you for all the number crunching to get all the OPR scores, Ether.

You're quite welcome.

The actual elapsed time to get from raw data to those reports for all 109 events (6398 partial OPR reports) was only 17 seconds on a single core of a Pentium D in an 8-year-old machine running XP Pro SP3, using AWK to wrangle the data and Octave to crunch the linear algebra.