OCCRA
Go to Post You really can't have a bad year. The only way you can is to come out of it and not learn anything. - InfernoX14 [more]
Home
Go Back   Chief Delphi > FIRST > General Forum
CD-Events   CD-Media   CD-Spy   FRC-Spy  
portal register members calendar search Today's Posts Mark Forums Read FAQ rules

 
Reply
Thread Tools Rate Thread Display Modes
  #1   Spotlight this post!  
Unread 03-27-2009, 12:44 AM
rsisk's Avatar
rsisk rsisk is offline
The GURU Channel
AKA: Richard Sisk
FRC #2493 (Robokong)
Team Role: Mentor
 
Join Date: Jan 2008
Rookie Year: 2007
Location: Riverside, CA
Posts: 2,447
rsisk has a reputation beyond reputersisk has a reputation beyond reputersisk has a reputation beyond reputersisk has a reputation beyond reputersisk has a reputation beyond reputersisk has a reputation beyond reputersisk has a reputation beyond reputersisk has a reputation beyond reputersisk has a reputation beyond reputersisk has a reputation beyond reputersisk has a reputation beyond repute
Send a message via MSN to rsisk
National Rankings

I have been gathering all of the Standings data in a database over the last few weeks of regionals and I would like to build a National Standings page, just like the regional standings page, but aggregating all the standings data together.

Some of the fields to include are pretty obvious, but I am having trouble figuring out how best to present some of the fields when teams play at more than one regional and I am hoping the statistics genius out there might lend a hand...

Rank -- Just like the regional, sort by QS, RS, then MP
Team -- Another no brainer, one entry per team
Wins -- Sum all wins
Losses -- Sum of all losses
Ties -- Sum of all ties
Played -- Sum of all matches played

Now here is where I get a bit lost...
QS -- Can't just do a sum because the more regionals, the higher this value would be. Maybe a weighted average? How best to calculate this?
RS -- Same thoughts as QS, maybe a weighted average?
MP -- I think this is easy, MAX(MP) from all regionals the team played

I would also add a new column...
Regionals -- Number of regionals where the team competed.


Any other data I should include?

Thanks in advance =
Reply With Quote
  #2   Spotlight this post!  
Unread 03-27-2009, 06:31 AM
davidfv davidfv is offline
Registered User
FRC #0399 (Eagle Robotics)
Team Role: Tactician
 
Join Date: Jan 2006
Rookie Year: 2004
Location: Lancaster, CA
Posts: 128
davidfv has a brilliant futuredavidfv has a brilliant futuredavidfv has a brilliant futuredavidfv has a brilliant futuredavidfv has a brilliant futuredavidfv has a brilliant futuredavidfv has a brilliant futuredavidfv has a brilliant futuredavidfv has a brilliant futuredavidfv has a brilliant futuredavidfv has a brilliant future
Re: National Rankings

I am not sure how to normalize the data per regional for just wins and losses. The teams that have participated in three regionals have more opportunity for wins and losses. One of the things that I have done is look at the average score per match.

If one team consistently has more points in every match no matter who the alliance partners happen to be, we might be able to assume that this was a result of a combination of coaching, strategy, team work, and scoring. For example, in LA Regional, using the twitter data, one team averaged about 85 points where the next teams were group at about 65. Call this a alliance score rating. Just some thoughts...
__________________
Davidfv
FRC Team 399 Lead Mentor/Drive Coach
Reply With Quote
  #3   Spotlight this post!  
Unread 03-27-2009, 07:08 AM
Alan Anderson's Avatar
Alan Anderson Alan Anderson is online now
Software Architect
FRC #0045 (TechnoKats)
Team Role: Mentor
 
Join Date: Feb 2004
Rookie Year: 2004
Location: Kokomo, Indiana
Posts: 7,817
Alan Anderson has a reputation beyond reputeAlan Anderson has a reputation beyond reputeAlan Anderson has a reputation beyond reputeAlan Anderson has a reputation beyond reputeAlan Anderson has a reputation beyond reputeAlan Anderson has a reputation beyond reputeAlan Anderson has a reputation beyond reputeAlan Anderson has a reputation beyond reputeAlan Anderson has a reputation beyond reputeAlan Anderson has a reputation beyond reputeAlan Anderson has a reputation beyond repute
Send a message via AIM to Alan Anderson
Re: National Rankings

Quote:
Originally Posted by rsisk View Post
QS -- Can't just do a sum because the more regionals, the higher this value would be. Maybe a weighted average? How best to calculate this?
Normalize it. Add them up and divide the sum by the total number of matches played to get a value from 0 to 2. Multiply by a large constant (e.g. 100) if you don't like fractions.

Quote:
RS -- Same thoughts as QS, maybe a weighted average?
The Ranking Score is already an average, so yeah. Multiply by the number of matches in the regional, add, then divide by the total number of matches played.
Reply With Quote
  #4   Spotlight this post!  
Unread 03-27-2009, 07:14 AM
Bongle's Avatar
Bongle Bongle is offline
Registered User
FRC #2702 (REBotics)
Team Role: Mentor
 
Join Date: Feb 2004
Rookie Year: 2002
Location: Waterloo
Posts: 1,016
Bongle has a reputation beyond reputeBongle has a reputation beyond reputeBongle has a reputation beyond reputeBongle has a reputation beyond reputeBongle has a reputation beyond reputeBongle has a reputation beyond reputeBongle has a reputation beyond reputeBongle has a reputation beyond reputeBongle has a reputation beyond reputeBongle has a reputation beyond reputeBongle has a reputation beyond repute
Send a message via MSN to Bongle
Re: National Rankings

Even with normalizing, I don't think the results are comparable. A win at the Midwest regional with its pile of top-end teams and multiple matches over 100 points is worth more than a win at, say, Waterloo. A team that went 4-4-0 at midwest might go 8-2-0 at Waterloo because of the lower level of competition there.

You could try ranking teams by OPR. That would be a bit more comparable between regionals.
Reply With Quote
  #5   Spotlight this post!  
Unread 03-27-2009, 09:55 AM
JesseK's Avatar
JesseK JesseK is offline
Flybotix Fanatic
FRC #1885 (iLITE)
Team Role: Mentor
 
Join Date: Mar 2007
Rookie Year: 2005
Location: Reston, VA
Posts: 2,803
JesseK has a reputation beyond reputeJesseK has a reputation beyond reputeJesseK has a reputation beyond reputeJesseK has a reputation beyond reputeJesseK has a reputation beyond reputeJesseK has a reputation beyond reputeJesseK has a reputation beyond reputeJesseK has a reputation beyond reputeJesseK has a reputation beyond reputeJesseK has a reputation beyond reputeJesseK has a reputation beyond repute
Re: National Rankings

What do you need your statistics to show? What will they be used for? Statistics for the sake of statistics usually means that there is some bias, hidden or invalid data, and/or bad predictions on the outcome of the statistics. In FRC, a team that plays at more than 1 regional is usually better than a team that plays at only 1 regional. In addition (as Bongle just mentioned) you have no strength of schedule weighted into any of this. Competition may be fierce with magnificent strategies and robots at one event, whereas robots at another event may barely move and the one that simply works the best wins.

When considering how to rank nation-wide OPR statistics, I had to consider the end application for the way I want to use the statistics. Right now, I want to somewhat predict every match in Atlanta based upon OPR, CCWM, and SOS statistics. So I decided to use only the data from a team's most recent regional, since many teams upgrade before Atlanta. However, if I were to see 'what robot is better' for the whole year (useful for FF picks), I would have to factor in all of a team's competitions. If the team was terrible at one regional then upgraded for the next regional and was great, that should factor in to my predictions for next year's teams' performance.
Reply With Quote
  #6   Spotlight this post!  
Unread 03-27-2009, 10:50 AM
EricH's Avatar
EricH EricH is offline
New year, new team
FRC #1197 (Torbots)
Team Role: Engineer
 
Join Date: Jan 2005
Rookie Year: 2003
Location: SoCal
Posts: 15,542
EricH has a reputation beyond reputeEricH has a reputation beyond reputeEricH has a reputation beyond reputeEricH has a reputation beyond reputeEricH has a reputation beyond reputeEricH has a reputation beyond reputeEricH has a reputation beyond reputeEricH has a reputation beyond reputeEricH has a reputation beyond reputeEricH has a reputation beyond reputeEricH has a reputation beyond repute
Send a message via Skype™ to EricH
Re: National Rankings

Quote:
Originally Posted by JesseK View Post
In addition (as Bongle just mentioned) you have no strength of schedule weighted into any of this.
The ranking score, which is based on the losing alliance's score, is in fact a strength of schedule rating. The higher it is, the tougher your schedule. Last year, if you had an RS of 80+, you had a very, very tough schedule. Even IRI only had one or two teams with it that high...one of which wound up winning the event.
__________________
Past teams:
2003-2007: FRC0330 BeachBots
2008: FRC1135 Shmoebotics
2012: FRC4046 Schroedinger's Dragons

Read the rules and make a picklist before thinking about winning a competition.

"Rockets are tricky..."--Elon Musk
Reply With Quote
  #7   Spotlight this post!  
Unread 03-27-2009, 11:32 AM
Lil' Lavery Lil' Lavery is offline
#8 Alliance For Life
AKA: Sean Lavery
FRC #1712 (DAWGMA)
Team Role: Mentor
 
Join Date: Nov 2003
Rookie Year: 2003
Location: Philadelphia, PA
Posts: 5,215
Lil' Lavery has a reputation beyond reputeLil' Lavery has a reputation beyond reputeLil' Lavery has a reputation beyond reputeLil' Lavery has a reputation beyond reputeLil' Lavery has a reputation beyond reputeLil' Lavery has a reputation beyond reputeLil' Lavery has a reputation beyond reputeLil' Lavery has a reputation beyond reputeLil' Lavery has a reputation beyond reputeLil' Lavery has a reputation beyond reputeLil' Lavery has a reputation beyond repute
Send a message via AIM to Lil' Lavery
Re: National Rankings

Is this being used for serious ranking, or just fun?

QS is quite simple. Win percentage . QS is a function of W-L-T anyway, so create a win% (with ties counting as .5 wins), and it would accomplish the same thing.


As a serious ranking, it simply isn't feasible. No currently existing metric, whether official FIRST or Chief Delphi-created, can give accurate rankings from different events (even ignoring the issues of sample size). Even OPR and DPRs are skewed by different opponents (a team is going to score a lot more at an event with more "easy targets").
If you could somehow create a complex algorithm based on teams who competed at multiple events to create a standard across those events, then maybe. But even then you run into problems with teams who change their robot, or otherwise get better or worse, from event to event. Israel literally has no crossover teams to be used, and many events have very few teams who compete at other regionals.
Reply With Quote
  #8   Spotlight this post!  
Unread 03-27-2009, 12:11 PM
rsisk's Avatar
rsisk rsisk is offline
The GURU Channel
AKA: Richard Sisk
FRC #2493 (Robokong)
Team Role: Mentor
 
Join Date: Jan 2008
Rookie Year: 2007
Location: Riverside, CA
Posts: 2,447
rsisk has a reputation beyond reputersisk has a reputation beyond reputersisk has a reputation beyond reputersisk has a reputation beyond reputersisk has a reputation beyond reputersisk has a reputation beyond reputersisk has a reputation beyond reputersisk has a reputation beyond reputersisk has a reputation beyond reputersisk has a reputation beyond reputersisk has a reputation beyond repute
Send a message via MSN to rsisk
Re: National Rankings

What I am trying to accomplish is the equivalent of the Regional ranking page for each regional but aggregating data from all the regionals.

If each team only competed in one regional, the task would be easy, I would just aggregate all the data. I need to deal with teams that compete in more than one regional, and I am leaning towards Alan's suggestion of normalization.

I'm not trying to correct for other variables such as alliance strength that the OPR type of algorithm would do, as far as I know ;>

And for the question of whether this is for fun or serious business, well, it's always for fun and I hope it produces something useful or interesting for others.
Reply With Quote
  #9   Spotlight this post!  
Unread 03-27-2009, 03:45 PM
AlexD744 AlexD744 is offline
Registered User
FRC #0744 (744 Shark Attack)
Team Role: Alumni
 
Join Date: Jan 2009
Rookie Year: 2008
Location: Ft. Lauderdale, FL
Posts: 600
AlexD744 has a reputation beyond reputeAlexD744 has a reputation beyond reputeAlexD744 has a reputation beyond reputeAlexD744 has a reputation beyond reputeAlexD744 has a reputation beyond reputeAlexD744 has a reputation beyond reputeAlexD744 has a reputation beyond reputeAlexD744 has a reputation beyond reputeAlexD744 has a reputation beyond reputeAlexD744 has a reputation beyond reputeAlexD744 has a reputation beyond repute
Re: National Rankings

Even if each team only competed at one regional, they still play different numbers of matches. Especially since some go to eliminations and other don't, unless your just basing this on qualifications. But even so a team at Florida played 9 matches in qualifyers, while a team in New Orleans may have played 12 or 13 (not sure of that number). I think an OPR over the whole nation would give you the closest things to an accurate ranking system. It would not be perfect but it would be close.
__________________
Phillipians 4:8 - Finally, brethren, whatever is true, whatever is honorable, whatever is right, whatever is pure, whatever is lovely, whatever is of good repute, if there is any excellence and if anything worthy of praise, dwell on these things.

God created FIRST as one of many wonderful things to dwell upon (just don't get obsessed ... whoops too late).
www.sharkattack744.com
Reply With Quote
  #10   Spotlight this post!  
Unread 03-27-2009, 03:49 PM
AndyB's Avatar
AndyB AndyB is offline
Ambitiously Disappointing
AKA: Andy Burchardt
FRC #0171 (Cheese Curd Herd)
Team Role: College Student
 
Join Date: Mar 2005
Rookie Year: 2004
Location: Platteville, WI
Posts: 1,185
AndyB has a reputation beyond reputeAndyB has a reputation beyond reputeAndyB has a reputation beyond reputeAndyB has a reputation beyond reputeAndyB has a reputation beyond reputeAndyB has a reputation beyond reputeAndyB has a reputation beyond reputeAndyB has a reputation beyond reputeAndyB has a reputation beyond reputeAndyB has a reputation beyond reputeAndyB has a reputation beyond repute
Re: National Rankings

I'm not intending to tear down your idea, because it's a good one, but team rankings aren't really a great way to value success or valuing one robot over another...

We already have the OPR rankings that get done which are quite possibly the best possible way to rank teams based solely on match scores.
__________________
Team 171 :: Cheese Curd Herd :: College Mentor, 2008-Present
Team 269 :: CooneyTech Robotics :: Student, 2005-2007
Reply With Quote
  #11   Spotlight this post!  
Unread 03-29-2009, 01:28 PM
Legator91's Avatar
Legator91 Legator91 is offline
Registered User
AKA: Ryan Legato
FRC #0067 (HOT Team)
Team Role: Alumni
 
Join Date: Mar 2009
Rookie Year: 2007
Location: Michigan
Posts: 21
Legator91 is a jewel in the roughLegator91 is a jewel in the roughLegator91 is a jewel in the roughLegator91 is a jewel in the rough
Send a message via AIM to Legator91
Re: National Rankings

Quote:
Originally Posted by AndyB View Post
I'm not intending to tear down your idea, because it's a good one, but team rankings aren't really a great way to value success or valuing one robot over another...

We already have the OPR rankings that get done which are quite possibly the best possible way to rank teams based solely on match scores.
yea I agree....a team could compete in 3 events and have a much higher win loss record then a team that only competed in 1 event. This doesn't mean the other team is bad, they just haven't competed in as many matches. I think it needs to be based more on how much each team and score and defend. rather then win loss record.
__________________
2010 World Champs (294, 67, 177)
2010 Michigan State Finalist
2009 World Champs (111, 67, 971)
2009 Michigan State Champs
Reply With Quote
  #12   Spotlight this post!  
Unread 03-29-2009, 01:38 PM
CSideris28's Avatar
CSideris28 CSideris28 is offline
Registered User
AKA: Christos Sideris
FRC #1155 (Sciborgs)
Team Role: Alumni
 
Join Date: Aug 2008
Rookie Year: 2007
Location: NY, NY
Posts: 27
CSideris28 will become famous soon enough
Send a message via AIM to CSideris28
Re: National Rankings

I disagree that RS can be used to show the strength of schedule, as a team that plays good defense will have a lower RS than one which does not.
__________________
2009 UTC Regional Champions!!! (Thanks 126 and 1902!!! Exploding Sciborg Force!!)
2009 NYC Regional Finalists (Thanks 2344 and 743!!!)
2009 NYC Regional Excellence in Automation and Control Award Winner
2007 NYC Regional Chairmans Award Winner
2007 NJ Regional Finalists (Thanks 359 and 486!!!)
Reply With Quote
  #13   Spotlight this post!  
Unread 03-29-2009, 03:43 PM
rsisk's Avatar
rsisk rsisk is offline
The GURU Channel
AKA: Richard Sisk
FRC #2493 (Robokong)
Team Role: Mentor
 
Join Date: Jan 2008
Rookie Year: 2007
Location: Riverside, CA
Posts: 2,447
rsisk has a reputation beyond reputersisk has a reputation beyond reputersisk has a reputation beyond reputersisk has a reputation beyond reputersisk has a reputation beyond reputersisk has a reputation beyond reputersisk has a reputation beyond reputersisk has a reputation beyond reputersisk has a reputation beyond reputersisk has a reputation beyond reputersisk has a reputation beyond repute
Send a message via MSN to rsisk
Re: National Rankings

So here is a question that may stimulate more discussion...

What does the Standings page on the USFirst.org web site show, which is the same as the display in the pits during the regional?

I know technically what it is showing, but in realtion to this discussion.

Some here say that OPR gives a better idea of the overall strength of the team and thus provides better scouting info. That makes sense.

But yet there is always a crowd around the display in the pits to see what position in the regional standing the team is in. Would it be as interesting to see the same information across all regionals?
Reply With Quote
  #14   Spotlight this post!  
Unread 04-01-2009, 09:46 AM
MikeReilly's Avatar
MikeReilly MikeReilly is offline
Registered User
FRC #4509 (Mechanical Bulls)
Team Role: Mentor
 
Join Date: Apr 2008
Rookie Year: 2005
Location: Sugar Hill, GA
Posts: 67
MikeReilly is a name known to allMikeReilly is a name known to allMikeReilly is a name known to allMikeReilly is a name known to allMikeReilly is a name known to allMikeReilly is a name known to all
Re: National Rankings

I'm into the SOS (strength of schedule) discussion. For example, at one recent regional, one team ended up ranked 6th and their wins included 3 no-shows. Another highly ranked team had no ability to score, just drive and deliver Empty Cells. (This is not a slam against any team, it's a mathematical discussion.)

So, what about this for SOS:
Using historical OPR, RS, or whatever stat is most relevant, go back and give Alliances theoretical score predictions. If Blue Alliance of 1-2-3 has an OPR of 100 versus the Red Alliance of 4-5-6 and their OPR of 20, 1-2-3 should win by 80. That would be an "easy" match for 1-2-3, a "hard" match for 4-5-6. Each alliance would get a 1 for "easy", 2 for "medium", 3 for "hard", and then average the match scores for their SOS.

You could go further, by using the difference of the Alliance OPR and the actual match score difference.

Just thinkin', maybe I'll get around to doing it.
__________________
Mentor for Rookie Team 4509, Formerly with 1771
Reply With Quote
Reply


Thread Tools
Display Modes Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
National Championship or National Chairman's Eric Bareiss General Forum 95 06-20-2005 09:07 PM
National Rankings during the Season Andrew Scouting 13 02-12-2004 02:17 PM
Rankings archiver 2001 3 06-24-2002 02:25 AM
NATIONAL RANKINGS! archiver 2001 1 06-24-2002 02:18 AM
NATIONAL RANKINGS... archiver 2001 0 06-24-2002 02:17 AM


All times are GMT -5. The time now is 11:18 PM.

The Chief Delphi Forums are sponsored by Innovation First International, Inc.


Powered by vBulletin® Version 3.6.4
Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Copyright © Chief Delphi