How many matches are really needed to determine final rankings?

[dougwilliams]

Originally Posted by Ether

...
If this looks interesting/promising, I will generate partial OPRs for all 106 events thus far (Weeks 1 through 6).
...

It does look interesting and I'd be interested in seeing that. I'd also be interested in seeing what code/spreadsheet magic both you, and the original poster are using to process that match data.

I put together something the other day that calculated my team's average after every Qual match through the VA regional. The above graphs are updated after each match round. I only needed something quick, and that seemed to be quicker, but probably not following the official scoring methodology.

[Dominick Ferone]

I believe that especially with this years game ranking can jump a lot. I know at the Tech Valley regional we "lost" a lot of matches with our score being lower then the opponent that match. But in the end it doesn't matter just of who has the best bot but who plays the alliance they have the smartest and most effective. Three of the lower ranked bots can succeed if played correctly and this year it seemed if teams figured out what worked best for them from the get go and kept to their strategy they would usually have a lot of success.

[Ether]

partial Final-Score OPRs for all 106 events weeks 1-6 can be found here:

http://www.chiefdelphi.com/media/papers/3125

[Conor Ryan]

I love this type of research, it really helps to tune those algorithmic scouting applications.

This type of work is incredibly similar to mathematical economics/econometrics type fields.

Fundamentally you'll see the law of averages dictate direction, but the real question is what was the Quality of the Other Alliance Members for each team based on the final ranking. The Strength of Schedule should be a good proxy for determining the quality of the match scheduling, which should help you determine the minimum number of matches needed to rank better.

But hey, we all have those days where we jump from 48th to 6th in the last 4 matches.

[tctc]

Fun comments. Thanks!

I decided to go ahead and look at the top seed alliance effect. That is, how does a team's rank change depending on if it is with or against the top seed team during a match, or not in a match at all with the top seed team? To do the analysis, I essentially took out every match that the top seed was in; marked which teams were with, against, or not in a match with the top seed; and then looked at how the new rankings changed versus the official rankings.

See below for the Silicon Valley (SV) and North Star (NS) Regionals. I took out team 254 from the SV Regional and 2826 from the NS Regional.



In the SV Regional, a team's overall rank increased by about 6.55+/-5.81 places if it was in an alliance with 254. But, a team's overall rank decreased by 5.87+/-3.49 places if the team did not have a match with 254.

In the NS Regional, a team's overall rank increased by about 8.05+/-8.68 places if it was in an alliance with 2826. But, a team's overall rank decreased by 5.79+/-2.58 places if the team did not have a match with 2826.

Ya, I would say this confirms the hypothesis that being in an alliance with the top team is going to boost a team's overall ranking. Likewise, not being in a match with the top team does not help.

Ether and Doug, I'll look into making some plots tomorrow with the OPR, including attaching the code for getting the data (just a heads up, it's not totally automated). But, ya, could be interesting.

[tctc]

Originally Posted by dougwilliams

It does look interesting and I'd be interested in seeing that. I'd also be interested in seeing what code/spreadsheet magic both you, and the original poster are using to process that match data.

See attached for the code (ImportResultsData.txt) and spreadsheet (FRC Team Rankings Chart.xlsx) for computing match data. To run it yourself, you'd need to import the code file into an Excel macro and save the Excel file as macro-enabled. Then set up the Match Results and Team Scores spreadsheets as described in the comments (you can also see other sheets in the Excel file for examples of how the data looks and how they are named). Then run the macro, and it should import team scores for each match. It doesn't get matches if the team was disqualified, unfortunately; so those will need to be updated manually. Then, compute average scores and rankings with Excel's average() and sorting functions. It is a bit of work by hand; but once one gets the hang of it, I think it goes pretty quickly. I hope this helps, and let me know if you have any questions.