Offensive Power Rankings for 2008

[JB987]

ss

Originally Posted by sumadin

I have the rankings generated from Greg Marra's csv dump of the TBA database.

Top 50:

1114	66.10642467
2056	54.11894734
233	50.73081127
330	48.61296306
987	48.40552663
39	48.06177403
525	45.48888639
67	44.23000818
1731	42.99226989
1124	42.40054838
217	42.10281885
103	41.29156933
40	40.86592771
175	39.4420245
191	39.06592804
469	37.90116922
20	37.75470786
79	37.65042852
1717	37.34613148
141	37.32830996
1024	37.2668216
968	35.93916079
1086	35.8662985
1126	35.81116812
368	35.65581359
494	35.0189302
2171	33.98399753
25	33.89508807
1625	33.79257371
365	33.63738509
254	32.91821392
1718	32.54432471
47	32.49881588
93	32.23816608
1477	32.14182287
1251	31.57996364
1806	31.33327422
16	31.18613493
69	31.04681657
1629	31.01151072
33	30.94122998
1418	30.31278837
195	30.2915486
27	29.90208154
383	29.89513167
71	29.87541039
287	29.76950247
171	29.71637571
148	29.4612061
877	29.34477514

The entire spreadsheet is attached. I'm working with Greg Marra on getting some more specific data sets and analyzing them, and after I do that, I'll post the results. Any specific requests?

Curious...what data or calculation changes explain the reduction of up to almost 3 points from the previous ranking for some teams listed? If the previous calculations were based on results through the 5th week regionals and there has been no additional results/data, what explains the changes observed for this list?

[Bongle]

Originally Posted by JB987

ss

Curious...what data or calculation changes explain the reduction of up to almost 3 points from the previous ranking for some teams listed? If the previous calculations were based on results through the 5th week regionals and there has been no additional results/data, what explains the changes observed for this list?

Since Sumadin and I went off of different data sets (him from TBA CSV's, me from usfirst.org) and used different algorithms, some perturbation is expected. There could've been a few missed matches in one of our data sets. This is my best guess for why there is a difference between our outputs.

Also, different methods of inverting the matrix could cause somewhat substantial differences in the floating-point input.

I haven't had the time to update my program to use TBA's CSV dump, so we won't be able to compare until probably tuesday when my classes end.

[Joe Ross]

Originally Posted by Bongle

Since Sumadin and I went off of different data sets (him from TBA CSV's, me from usfirst.org) and used different algorithms, some perturbation is expected. There could've been a few missed matches in one of our data sets. This is my best guess for why there is a difference between our outputs.

The Blue Alliance data includes elimination matches, which I would assume accounts for the majority of the difference. I think the result is probably better with only the qualifying matches.

The Blue Alliance data also gives scores of -1 for the missing peachtree matches (which are now in TBA database, so they would be included for the next data dump).

I've attached just the GTR qualifying matches (which I pulled from TBA's big dump) so that it is possible for sumadin to check his results against Bongle and Kyler's.

[IndySam]

Originally Posted by Bongle

It is an estimate of how many points your teams contributes per match. Note that it isn't simply your total score / 3, but by using the magic of linear algebra, you can solve for something a bit more accurate. It can help weed out robots that simply had good partners, and should be able to discern robots who many not have placed well but still contributed each round.

Our actual average score was 23.11 for our two regionals including penalties. You have us at 13.585. That's a huge difference.

I also see a couple of teams on your list that I know who's scores are way to high.

As I have said from the beginning the algorithm just won't work unless the penalties are somehow added in. They are just to important to this years game.

[Karthik]

Originally Posted by IndySam

Our actual average score was 23.11 for our two regionals including penalties. You have us at 13.585. That's a huge difference.

I also see a couple of teams on your list that I know who's scores are way to high.

As I have said from the beginning the algorithm just won't work unless the penalties are somehow added in. They are just to important to this years game.

I don't think you're following what Bongle and others are doing. They're not calculating simple averages. They're using basic linear algebra to calculate the average expected value of pure points contributed to the alliance by a single team. This is used by setting up a system of equations which sum to the amount points scored at the regional. Since penalties are already removed from the final scores of all matches, this method takes them into account. The penalties are already "added in". This method is a far more accurate predictor of a team's contribution then just looking at a simple mean. Naturally you team's result is going to be lower than your average score, because this statistic is just looking at your team's contribution.

[Guy Davidson]

Originally Posted by Joe Ross

I've attached just the GTR qualifying matches (which I pulled from TBA's big dump) so that it is possible for sumadin to check his results against Bongle and Kyler's.

Here are my GTR numbers. If they look wrong, and someone java-literate feels like checking my algorithm, I'd love to send it to you.

Team #	 OPR
1114	85.15226558
2056	59.33395475
1310	34.64482448
703	33.03214195
176	28.96394877
1507	28.88990882
843	28.21126973
1565	27.22743171
247	26.1033492
1334	25.79040538
1246	25.39034964
771	25.3041279
2166	24.09227148
188	23.71808394
1503	23.54927373
772	23.45548442
2386	23.31079522
1241	23.28145949
1676	19.79855182
2200	18.77658592
2505	17.48254181
1859	17.45000723
781	16.0161082
1075	15.90296028
1305	15.89858958
854	15.69463168
1141	15.06495691
1870	14.73720734
2198	14.73609737
378	14.47812438
1482	14.45778707
1221	14.31262296
2624	13.45441951
1547	12.97880742
1605	12.97081727
1312	12.72903502
1053	12.31981296
2625	10.56083704
2013	9.900241288
1244	9.708875262
1564	9.600009496
1404	9.377591833
610	9.133722417
2626	9.08257019
919	8.254425484
1006	7.228797019
1620	6.76173277
1219	6.42921342
1325	6.212867948
907	5.392902881
2670	5.28688774
1835	4.499976969
2634	2.964469403
1814	2.838298002
1514	2.750910246
2076	1.743258698
1846	0.608947798
1815	-1.060063643
1558	-2.177781773
865	-3.154743822
2427	-4.468775757
2254	-5.065213583
296	-5.98087571
758	-6.886956522
1680	-8.189157537
2185	-9.563978454

[Karthik]

Originally Posted by sumadin

Here are my GTR numbers. If they look wrong, and someone java-literate feels like checking my algorithm, I'd love to send it to you.

Guy,

You might want to look into using Maple or Matlab. They're design for these types of computation, plus I'm sure you'll find plenty of other uses for them. I've found them to be indispensable tools.

[Joe Ross]

Originally Posted by sumadin

Here are my GTR numbers. If they look wrong, and someone java-literate feels like checking my algorithm, I'd love to send it to you.

They are exactly the same as Bongle's to 4 decimal places. Good job!

[Guy Davidson]

Originally Posted by JB987

Curious...what data or calculation changes explain the reduction of up to almost 3 points from the previous ranking for some teams listed? If the previous calculations were based on results through the 5th week regionals and there has been no additional results/data, what explains the changes observed for this list?

I was running the TBA data set. It seems that my numbers for GTR match Bongle's so the computation seems to be correct. Is it possible he isn't using elimination round scores? Because that might cause a drop.

[Bongle]

Originally Posted by sumadin

I was running the TBA data set. It seems that my numbers for GTR match Bongle's so the computation seems to be correct. Is it possible he isn't using elimination round scores? Because that might cause a drop.

Yeah, I didn't use elimination rounds because I feel they kind of distort the results. Playing matches with random robots against random opponents, I think you'll get a better feel for a team's contribution than when they are playing with a self-selected team against the creme de la creme at the regional. Although I guess if you want to take these numbers as scouting pointers rather than for entertainment value, a team's elimination performance is important.

For instance, take 1114 at GTR in the eliminators: they were playing with a team that they founded and they had won 3 regionals with. That team of robots will probably work together much better than a random team in qualifications, and so the scores will be different.

Another thing is that during eliminations, robots steadily get worn down: mechanisms break that can't get repaired in time, batteries slowly get discharged, motors overheat, and so you get things other than the team's competitiveness affecting the scoring level.

Plus, at a shallower regional like at waterloo, you'll end up with things like 1114/2056 against 3 teams who can't even hurdle, and the scores will be ludicrously high.

I don't think you're following what Bongle and others are doing. They're not calculating simple averages. They're using basic linear algebra to calculate the average expected value of pure points contributed to the alliance by a single team. This is used by setting up a system of equations which sum to the amount points scored at the regional. Since penalties are already removed from the final scores of all matches, this method takes them into account. The penalties are already "added in". This method is a far more accurate predictor of a team's contribution then just looking at a simple mean. Naturally you team's result is going to be lower than your average score, because this statistic is just looking at your team's contribution.

Said much better than I ever could.

[IndySam]

Originally Posted by Karthik

I don't think you're following what Bongle and others are doing. They're not calculating simple averages. They're using basic linear algebra to calculate the average expected value of pure points contributed to the alliance by a single team. This is used by setting up a system of equations which sum to the amount points scored at the regional. Since penalties are already removed from the final scores of all matches, this method takes them into account. The penalties are already "added in". This method is a far more accurate predictor of a team's contribution then just looking at a simple mean. Naturally you team's result is going to be lower than your average score, because this statistic is just looking at your team's contribution.

I understand that, what I am saying is that their calculation of my teams expected contribution is no where near my teams actual contribution including subtracting the penalties.

Also I see several teams who's expected contribution is way to high from my experience of their performance this year.

If you guys want this tool to be useful you need to figure out why.

It's just my theory that it can't property calculate the impact of penalties, if you think this theory is wrong then you need to figure out what is causing the figure to be so different than the actual results.

[Karthik]

Originally Posted by IndySam

I understand that, what I am saying is that their calculation of my teams expected contribution is no where near my teams actual contribution including subtracting the penalties.

Also I see several teams who's expected contribution is way to high from my experience of their performance this year.

If you guys want this tool to be useful you need to figure out why.

It's just my theory that it can't property calculate the impact of penalties, if you think this theory is wrong then you need to figure out what is causing the figure to be so different than the actual results.

Mathematically, this method is 100% sound. Whether you see any value in the mathematical model is for you to determine. I'd like to know your mathematical reasoning for why this method doesn't take penalties into account. If all you're relying on is selected cases of anecdotal evidence, your theory holds very little weight with me.

By no means am I suggesting that this model is perfect. From the numbers that Team 1114 has run, there are some definite anomalies. That being said, for the most part this model seems to agree with the specific match data we've collected at three events we attended.

Like with most statistics, the value is all in how you interpret them. If you don't think they're accurate, ignore them.

[Mr. Lim]

Just to emphasize the quality of OPR as a predictor of who will be playing on Saturday afternoon. Keep in mind, this was a 66 team regional, and predicting 24 robots out of the field is no small feat.

Here's the top 24 OPR from the GTR, and whether they were picked in the elims:

1114	#1 Captain
2056	#1 1st Pick
1310	#2 1st Pick
703	Removed themselves from the elims due to technical problems
176	#7 Captain
1507	#3 Captain
843	#5 Captain
1565	#4 1st Pick
247	#3 2nd Pick
1334	#6 Captain
1246	Not Picked
771	#5 2nd Pick
2166	#1 2nd Pick
188	#2 Captain
1503	#3 1st Pick
772	#8 1st Pick
2386	#6 1st Pick
1241	#4 Captain
1676	#7 1st Pick
2200	#5 1st Pick
2505	#2 2nd Pick
1859	Not Picked
781	#4 2nd Pick
1075	Not Picked

20/24 is a ridiculously decent hit rate. It should be higher since you couldn't expect this algorithm to predict that 703 would graciously withdraw.

Expect all good scouting teams to be using these stats in the very near future.

[IndySam]

Originally Posted by Karthik

Mathematically, this method is 100% sound. Whether you see any value in the mathematical model is for you to determine. I'd like to know your mathematical reasoning for why this method doesn't take penalties into account. If all you're relying on is selected cases of anecdotal evidence, your theory holds very little weight with me.

By no means am I suggesting that this model is perfect. From the numbers that Team 1114 has run, there are some definite anomalies. That being said, for the most part this model seems to agree with the specific match data we've collected at three events we attended.

Like with most statistics, the value is all in how you interpret them. If you don't think they're accurate, ignore them.

I am not a mathematician, I saw numbers that were wrong (not a little wrong, WILDLY wrong) and I posed a theory based on my feeling not on any math.

We are not competing in Atlanta so I have no need for this data, all I am saying some of what I see in spreadsheet is very wrong.

You can ignore it, I don't care.


Edit: I'll still buy you a doughnut in Atlanta.

[Protronie]

All I understand is 1114 has an impressive score...
Which seems to fit their impressive robot.

Nice work... this will make a excellent tool to help newbies .