Chief Delphi

Chief Delphi (http://www.chiefdelphi.com/forums/index.php)
-   General Forum (http://www.chiefdelphi.com/forums/forumdisplay.php?f=16)
-   -   Easy to use Offensive Power Rankings (OPR) program for mid-regional scouting (http://www.chiefdelphi.com/forums/showthread.php?t=75272)

Killraine 03-03-2009 13:17

Re: Easy to use Offensive Power Rankings (OPR) program for mid-regional scouting
 
Also, since the scores are getting sorted anyway, could you add an actual number rank (ie highest opr would be 1, next would be 2, etc.)?

Bongle 04-03-2009 19:12

Re: Easy to use Offensive Power Rankings (OPR) program for mid-regional scouting
 
2 Attachment(s)
v6:

-Sorts OPR, SAA, and PMs in a more logical manner
-New 'all' statistic type that prints out OPR, SAA, and PM in a table. Different output for the console window versus command-line running so that you guys running it for excel will get your tabs, while the console-window people will still be able to read it.

Clinton Bolinger 05-03-2009 11:21

Re: Easy to use Offensive Power Rankings (OPR) program for mid-regional scouting
 
Looks good...

When you try the following the Pos is just in numerical order:

Code:

OPRNet.exe il 2009 all t q > all.txt
Any chance you can have it show their Rank instead?

-Oris-

Bongle 05-03-2009 11:43

Re: Easy to use Offensive Power Rankings (OPR) program for mid-regional scouting
 
Quote:

Originally Posted by Oris (Post 832108)
Looks good...

When you try the following the Pos is just in numerical order:

Code:

OPRNet.exe il 2009 all t q > all.txt
Any chance you can have it show their Rank instead?

-Oris-

Rather than 't' (for sort-by-team), use 'r' (for sort-by-rank). That'll sort the table by OPR first, though if you want to sort it, I'd recommend copying the contents of all.txt into excel and sorting there.

Killraine 05-03-2009 12:40

Re: Easy to use Offensive Power Rankings (OPR) program for mid-regional scouting
 
When I sort +/- by rank, the highest ranked team is rank "0" rather than "1"

Edit: Also, it would be nice to be able to sort by team, but still see what rank the teams are when sorted by rating. Ie. Sort by rating first to get a rank value for each team, then resort by team number. I know its not really necessary if you just take the data and put it in excel, but it would be nice for the program as a stand alone.

JesseK 05-03-2009 13:07

Re: Easy to use Offensive Power Rankings (OPR) program for mid-regional scouting
 
Bongle -- do you mind if I port this over to Java? I want to try to get a real-time OPR Scoring program working by the time Atlanta rolls around.

Bongle 05-03-2009 13:16

Re: Easy to use Offensive Power Rankings (OPR) program for mid-regional scouting
 
Quote:

Originally Posted by JesseK (Post 832147)
Bongle -- do you mind if I port this over to Java? I want to try to get a real-time OPR Scoring program working by the time Atlanta rolls around.

Go ahead. If you're running it from a server, I think it'd be easiest to just run oprnet once each match is done and parse the results. Porting the math stuff might be hard.

SteveGPage 06-03-2009 09:56

Re: Easy to use Offensive Power Rankings (OPR) program for mid-regional scouting
 
I have a general question that I would love for you all to consider. I am a firm believer in the OPR/DPR concept, and included it in our team's scouting application. But what I found was that in this game, ALLIANCE data in a match is not that helpful. If a really good team has an alliance with two essentially non-functional robots in 2 or 3 of your matches - the numbers look terrible for that team. A case in point, 234 (Cyberblue) at the DC regional - if you look at the OPR numbers for them, they are in the middle of the pack. Our scouting team did NOT look at the alliance data this year. I had scouts answer the following questions:
1) How many MR did they place in a trailer (including their own in the event they were trying to avoid the penalty)
2) How many MR were in their trailer (by either the opposing alliance, or the HP)
3) How many EC did they deliver to be exchanged with a SC

I then calculated the OPR and DPR on those stats. In my example above, regarding 234 - they jumped from middle of the pack, to essentially the same level as 45, which is where they should have been ranked.

Any one else looking at TEAM OPR and DPR, and if so, how are you doing it, and what were your results like.

By the way, when I looked at the predictive nature of the values we calculated, we were right in our predictions more than 85% of the time. Only something like "forgetting to turn on the radio" or some such issue would throw off the results.

Best regards,

Steve

Bongle 06-03-2009 10:11

Re: Easy to use Offensive Power Rankings (OPR) program for mid-regional scouting
 
Quote:

Originally Posted by SteveGPage (Post 832462)
I have a general question that I would love for you all to consider. I am a firm believer in the OPR/DPR concept, and included it in our team's scouting application. But what I found was that in this game, ALLIANCE data in a match is not that helpful.

Any team that scouts entirely off of the output of this program (or any OPR program) is doing something wrong. Personally, I use it to get a quick idea of which robots to watch if I'm watching a webcast and haven't been following an entire regional. I completely second your recommendation of in-person scouting. Just like the FRC rankings system often fails an otherwise good team, the OPR ranking algorithm can screw up too.

Quote:

If a really good team has an alliance with two essentially non-functional robots in 2 or 3 of your matches - the numbers look terrible for that team.
That's not necessarily true. If your two non-functional alliance partners are consistently non-functional, then your OPR should be unaffected. The main thing that throws off OPR is inconsistency. If an very good team has a battery fall out during a match, then it affects their alliance partners' OPR scores quite badly, because it 'looks' to the algorithm like their presence resulted in a massive reduction in their alliance's score.

Killraine 06-03-2009 11:27

Re: Easy to use Offensive Power Rankings (OPR) program for mid-regional scouting
 
If only we could have some kind of "fluke" marker that we could place on team's whenever they behave inconsitently. I don't know your algorithm, but maybe it could treat a fluke robot performance as a no-name default robot that performs at the low level it does.

JesseK 06-03-2009 13:58

Re: Easy to use Offensive Power Rankings (OPR) program for mid-regional scouting
 
Surprisingly, I got alot of the backbone of the porting done last night. Luckily Jama makes a Java library as well. I may have a first pass at it completed by the end of the weekend. All I have to do is figure out this silly URL issue...

Manoel 06-03-2009 14:14

Re: Easy to use Offensive Power Rankings (OPR) program for mid-regional scouting
 
Quote:

Originally Posted by JesseK (Post 832540)
Surprisingly, I got alot of the backbone of the porting done last night. Luckily Jama makes a Java library as well. I may have a first pass at it completed by the end of the weekend. All I have to do is figure out this silly URL issue...

Oh, how I understand you! ;)

I'm working on a MATLAB version since last night and the math is all working IF you input the data (copy from FIRST to Excel then to MATLAB, really easy but cumbersome). Coding to parse the results from FIRST's website is going to take the full weekend, I believe.

Bongle 06-03-2009 14:20

Re: Easy to use Offensive Power Rankings (OPR) program for mid-regional scouting
 
Quote:

Originally Posted by Manoel (Post 832546)
Oh, how I understand you! ;)

I'm working on a MATLAB version since last night and the math is all working IF you input the data (copy from FIRST to Excel then to MATLAB, really easy but cumbersome). Coding to parse the results from FIRST's website is going to take the full weekend, I believe.

That depends on how flexible you want your parsing to be. I spent 2 hours trying to write an uber-XML parser with MSXML, then I gave up and spent about half an hour hacking together the one that I use now. All of FIRST's score pages from mid-2008 onwards are generated from the same application, and have identical formatting. I just read one line at a time and looked for the text that always preceded a team score. It works well enough for me, though it'll need an update if they ever change their match results page style.

Better idea, if you're not already committed to scraping from FIRST: scrape from TBA - they use the same style for all events, all years. Your program would be able to run OPRs all the way back to 2006.

Edit: Speaking of parsing, I notice that the prediction code doesn't work with a regional underway. I'll have to look at that tonight (I wasn't able to properly test it since all regionals were done when I initially wrote that feature).

Manoel 08-03-2009 00:31

Re: Easy to use Offensive Power Rankings (OPR) program for mid-regional scouting
 
1 Attachment(s)
Quote:

Originally Posted by Bongle (Post 832548)
That depends on how flexible you want your parsing to be. I spent 2 hours trying to write an uber-XML parser with MSXML, then I gave up and spent about half an hour hacking together the one that I use now. All of FIRST's score pages from mid-2008 onwards are generated from the same application, and have identical formatting. I just read one line at a time and looked for the text that always preceded a team score. It works well enough for me, though it'll need an update if they ever change their match results page style.

Better idea, if you're not already committed to scraping from FIRST: scrape from TBA - they use the same style for all events, all years. Your program would be able to run OPRs all the way back to 2006.

Edit: Speaking of parsing, I notice that the prediction code doesn't work with a regional underway. I'll have to look at that tonight (I wasn't able to properly test it since all regionals were done when I initially wrote that feature).

You, sir, are wise. ;)

As a professor used to say, there are always at least two ways to solve a problem. More often than not, the easy ways are not correct and the correct ways are not easy. We chose easy, so eventually it will come and bite us.

I used TBA data so you can get stats for events from 2006 to 2009. It's nearly impossible to obtain a 2007 event matrix that's not ill-conditioned - I believe it's due to FIRST's awful alliance sorting algorithms from that year, but have not looked into in detail. In short, OPR for 2007 will not work. If someone is feeling reminiscent this days, please take a look.

Here's the code, just run OPR from MATLAB's command window and ask for what you want. The script returns three arrays containing ranked OPR, DPR (or whatever you might call it) and +/- stats for that particular event.

I have tested it with several events, but bugs may arise. Please test and let me know.

Bongle 08-03-2009 09:31

Re: Easy to use Offensive Power Rankings (OPR) program for mid-regional scouting
 
Quote:

Originally Posted by Manoel (Post 832967)
You, sir, are wise. ;)

As a professor used to say, there are always at least two ways to solve a problem. More often than not, the easy ways are not correct and the correct ways are not easy. We chose easy, so eventually it will come and bite us.

I used TBA data so you can get stats for events from 2006 to 2009. It's nearly impossible to obtain a 2007 event matrix that's not ill-conditioned - I believe it's due to FIRST's awful alliance sorting algorithms from that year, but have not looked into in detail. In short, OPR for 2007 will not work. If someone is feeling reminiscent this days, please take a look.

Here's the code, just run OPR from MATLAB's command window and ask for what you want. The script returns three arrays containing ranked OPR, DPR (or whatever you might call it) and +/- stats for that particular event.

I have tested it with several events, but bugs may arise. Please test and let me know.

It is probably a combination of the terrible alliance picking, and that 2007 had non-linear scoring. A robot that always scored 3 ringers in 2007 might contribute 6 points in one match (by scoring 2 points, 3 times), or it might contribute 192 points in another, by extending a row of 5 into a row of 8.

In 2006, 2008, and 2009, a robot that did <game action> 5 times in each match generally generated the same number of points.


All times are GMT -5. The time now is 00:38.

Powered by vBulletin® Version 3.6.4
Copyright ©2000 - 2017, Jelsoft Enterprises Ltd.
Copyright © Chief Delphi