OCCRA
Go to Post Woody is wearing flippers... 2014 WATER GAME! - Team23pitboss [more]
Home
Go Back   Chief Delphi > FIRST > General Forum
CD-Events   CD-Media   CD-Spy   FRC-Spy  
portal register members calendar search Today's Posts Mark Forums Read FAQ rules

 
Reply
Thread Tools Rate Thread Display Modes
  #1   Spotlight this post!  
Unread 04-13-2013, 10:39 PM
Ed Law's Avatar
Ed Law Ed Law is offline
Registered User
FRC #2834 (Bionic Black Hawks)
Team Role: Coach
 
Join Date: Apr 2008
Rookie Year: 2009
Location: Bloomfield Hills, Michigan, USA
Posts: 607
Ed Law has a reputation beyond reputeEd Law has a reputation beyond reputeEd Law has a reputation beyond reputeEd Law has a reputation beyond reputeEd Law has a reputation beyond reputeEd Law has a reputation beyond reputeEd Law has a reputation beyond reputeEd Law has a reputation beyond reputeEd Law has a reputation beyond reputeEd Law has a reputation beyond reputeEd Law has a reputation beyond repute
OPR after Week Seven Events

The OPR/CCWM numbers up to Week 7 events have been posted, please see

http://www.chiefdelphi.com/media/papers/2174

If you find any error or have any questions, please let me know.

I will post this again after the division data of the World Championship is announced.
__________________
Please don't call me Mr. Ed, I am not a talking horse.
Reply With Quote
  #2   Spotlight this post!  
Unread 04-13-2013, 11:46 PM
Grim Tuesday's Avatar
Grim Tuesday Grim Tuesday is offline
Registered User
AKA: Simon Bohn
FRC #0639 (Code Red)
Team Role: Alumni
 
Join Date: Jan 2010
Rookie Year: 2010
Location: Baltimore MD (JHU)
Posts: 1,575
Grim Tuesday has a reputation beyond reputeGrim Tuesday has a reputation beyond reputeGrim Tuesday has a reputation beyond reputeGrim Tuesday has a reputation beyond reputeGrim Tuesday has a reputation beyond reputeGrim Tuesday has a reputation beyond reputeGrim Tuesday has a reputation beyond reputeGrim Tuesday has a reputation beyond reputeGrim Tuesday has a reputation beyond reputeGrim Tuesday has a reputation beyond reputeGrim Tuesday has a reputation beyond repute
Re: OPR after Week Seven Events

So after 7 weeks, what is the final verdict on OPR in 2013? I feel like it's a pretty good measurement as scoring is mostly linear.
__________________

-Buckeye Regional Champion 2013, 2014
-Newton Division Finalists 2012 thanks to 68 and 330
Reply With Quote
  #3   Spotlight this post!  
Unread 04-14-2013, 02:53 AM
PayneTrain's Avatar
PayneTrain PayneTrain is offline
Unprepared
AKA: Wil Payne (@Wil_Payne)
FRC #0422 (The Mech Tech Dragons)
Team Role: Mentor
 
Join Date: Mar 2010
Rookie Year: 2008
Location: RVA
Posts: 1,231
PayneTrain has a reputation beyond reputePayneTrain has a reputation beyond reputePayneTrain has a reputation beyond reputePayneTrain has a reputation beyond reputePayneTrain has a reputation beyond reputePayneTrain has a reputation beyond reputePayneTrain has a reputation beyond reputePayneTrain has a reputation beyond reputePayneTrain has a reputation beyond reputePayneTrain has a reputation beyond reputePayneTrain has a reputation beyond repute
Send a message via Skype™ to PayneTrain
Re: OPR after Week Seven Events

Quote:
Originally Posted by Grim Tuesday View Post
So after 7 weeks, what is the final verdict on OPR in 2013? I feel like it's a pretty good measurement as scoring is mostly linear.
No one's totally sure until Karthik either praises or denigrates it in his strategy presentation this year.
__________________
Don't follow your dreams; follow my Twitter!
Reply With Quote
  #4   Spotlight this post!  
Unread 04-14-2013, 06:43 AM
gabrielau23's Avatar
gabrielau23 gabrielau23 is offline
Registered User
FRC #2537
Team Role: Leadership
 
Join Date: Oct 2012
Rookie Year: 2012
Location: MD
Posts: 174
gabrielau23 is just really nicegabrielau23 is just really nicegabrielau23 is just really nicegabrielau23 is just really nice
Re: OPR after Week Seven Events

I'm not sure how it's calculated, but I feel like it's a decent way to scout as long as it's followed up by actual scouting of a team. It's going to sound like I'm whining here, but it's true. Except for one match, our team scored above 50% (in some cases we scored above 65%) of our alliance's total points during the competition (including elims). Our average points per match was 40 (and this is including a match where we jammed and only scored 25 points). If we take that match out, I believe that we averaged over 44 points per match. Yet our team only won two matches during the entire competition. So, I mean, it's a good measurement, but teams are going to be missed.
Generally, however, I do think it's an invaluable tool for smaller teams who don't have the resources to do a bunch of scouting themselves. For the most part, robots can be measured by those metrics and it's mostly accurate.
__________________
The higher I go up, the the faster the best times in my life (like build season) pass...so sad.
Reply With Quote
  #5   Spotlight this post!  
Unread 04-14-2013, 07:35 AM
Bongle's Avatar
Bongle Bongle is offline
Registered User
FRC #2702 (REBotics)
Team Role: Mentor
 
Join Date: Feb 2004
Rookie Year: 2002
Location: Waterloo
Posts: 1,016
Bongle has a reputation beyond reputeBongle has a reputation beyond reputeBongle has a reputation beyond reputeBongle has a reputation beyond reputeBongle has a reputation beyond reputeBongle has a reputation beyond reputeBongle has a reputation beyond reputeBongle has a reputation beyond reputeBongle has a reputation beyond reputeBongle has a reputation beyond reputeBongle has a reputation beyond repute
Send a message via MSN to Bongle
Re: OPR after Week Seven Events

Quote:
Originally Posted by PayneTrain View Post
No one's totally sure until Karthik either praises or denigrates it in his strategy presentation this year.
I can think of other issues with OPR that are unchanging from year to year:
-How valuable is a scouting system that everyone else is using? How do you differentiate your picks from other teams' picks if you're all picking from the same OPR/CCWM-derived list?
-OPR doesn't take into account a robot's type and how it interacts with other robots. You can get a high OPR in qualifying simply by supplying discs to a floor loader that then scores them for you. But an alliance of disc-suppliers would score very poorly in eliminations.
-OPR doesn't compare well at all across events. Waterloo had a ridiculous average OPR (even excluding 1114/2056), but that was probably because nearly every robot there was offensive, and so little defense was played. If you added 10 defensive robots to the pool, everyone else's OPR would have fallen drastically. You can do a "global OPR", but I doubt it corrects that much.

OPR seemed to do very well this year as far as its ability to reflect robot's effectiveness within a regional, but there are broader issues with it than just that. IIRC, Karthik's main complaint about OPR was that people were using it for scouting and comparing robots without really understanding it or understanding its shortcomings like those I listed above.
Reply With Quote
  #6   Spotlight this post!  
Unread 04-14-2013, 08:37 AM
PayneTrain's Avatar
PayneTrain PayneTrain is offline
Unprepared
AKA: Wil Payne (@Wil_Payne)
FRC #0422 (The Mech Tech Dragons)
Team Role: Mentor
 
Join Date: Mar 2010
Rookie Year: 2008
Location: RVA
Posts: 1,231
PayneTrain has a reputation beyond reputePayneTrain has a reputation beyond reputePayneTrain has a reputation beyond reputePayneTrain has a reputation beyond reputePayneTrain has a reputation beyond reputePayneTrain has a reputation beyond reputePayneTrain has a reputation beyond reputePayneTrain has a reputation beyond reputePayneTrain has a reputation beyond reputePayneTrain has a reputation beyond reputePayneTrain has a reputation beyond repute
Send a message via Skype™ to PayneTrain
Re: OPR after Week Seven Events

^It was just a joke, bro. If a team bases scouting off of a singular statistical value (that they may not even fully understand).

I would say OPR is really good at figuring out at what level a robot can play at, but without the context of knowing the "class" of the robot, the number is meaningless. Teams 11 and 245 are some of the better known "cyclers" that may have a high OPR, but would a cycler pick another cycler over a floor loader because the cycler has a higher OPR?

I guess they always could, but that probably means scouting wants to take the rest of the day off and not worry about scouting any elims.

But maybe the cycler also played some impeccable defense and was very agile in transition so they could easily switch between offense and defense... and the rabbit hole get's deeper.
__________________
Don't follow your dreams; follow my Twitter!
Reply With Quote
  #7   Spotlight this post!  
Unread 04-14-2013, 09:33 AM
GearsOfFury's Avatar
GearsOfFury GearsOfFury is offline
Registered User
AKA: Mark Anderson
FRC #2338 (Gear It Forward)
Team Role: Mentor
 
Join Date: Jan 2008
Rookie Year: 2006
Location: Oswego, IL
Posts: 86
GearsOfFury is a splendid one to beholdGearsOfFury is a splendid one to beholdGearsOfFury is a splendid one to beholdGearsOfFury is a splendid one to beholdGearsOfFury is a splendid one to beholdGearsOfFury is a splendid one to beholdGearsOfFury is a splendid one to behold
Quote:
Originally Posted by Bongle View Post
I can think of other issues with OPR that are unchanging from year to year:
-How valuable is a scouting system that everyone else is using? How do you differentiate your picks from other teams' picks if you're all picking from the same OPR/CCWM-derived list?
Totally agree with all of your other points except this one. Why do you need to "differentiate your picks" for the sake of differentiation? You need to find the best robots available to complete your alliance. If there were a single, universal system to do this, it wouldn't be any less valuable just because everyone else was using it.
Reply With Quote
  #8   Spotlight this post!  
Unread 04-14-2013, 09:38 AM
Chris is me's Avatar
Chris is me Chris is me is offline
Work harder, do better.
AKA: Pinecone
FRC #2791 (Shaker Robotics) (EWCP)
Team Role: Mentor
 
Join Date: Dec 2008
Rookie Year: 2006
Location: Troy, NY
Posts: 6,532
Chris is me has a reputation beyond reputeChris is me has a reputation beyond reputeChris is me has a reputation beyond reputeChris is me has a reputation beyond reputeChris is me has a reputation beyond reputeChris is me has a reputation beyond reputeChris is me has a reputation beyond reputeChris is me has a reputation beyond reputeChris is me has a reputation beyond reputeChris is me has a reputation beyond reputeChris is me has a reputation beyond repute
Send a message via AIM to Chris is me
Re: OPR after Week Seven Events

OPR (specifically CCWM) is better than I expected at separating teams, and seems to be ~85% accurate based on a comparison to a few events of data.

That said, you have to remember that OPR is an approximation of skill that is better than average score but no replacement for a real scouting system. It's a decent "sanity check" to see if you missed an outlier or two but if you have actual data I wouldn't even touch it.

During and after the season, it's a very good approximation of skill that is very useful for Internet debates.
__________________
Proud 2791 Mentor (2010-201?):
2014 FLR Motorola Quality / Semifinalists (with 341, 4930)
2013 WPI Regional #1 Seed / Delphi Excellence in Engineering / Finalists (with 20, 3182), BAE Motorola Quality
2012 BAE Imagery / Finalists (with 1519, 885), CT Xerox Creativity / Semifinalists (with 2168, 118)

Proud 1714 Alumnus (2009) - 2009 Minnesota 10,000 Lakes Regional Champions (with 2826, 2470)
2791 Build Season Photo Gallery - Look here for mechanism photos
My Robotics Blog (Updated April 11 2014)
Reply With Quote
  #9   Spotlight this post!  
Unread 04-14-2013, 10:24 AM
Tom Line's Avatar
Tom Line Tom Line is offline
Raptors can't turn doorknobs.
FRC #1718 (The Fighting Pi)
Team Role: Mentor
 
Join Date: Jan 2007
Rookie Year: 1999
Location: Armada, Michigan
Posts: 2,139
Tom Line has a reputation beyond reputeTom Line has a reputation beyond reputeTom Line has a reputation beyond reputeTom Line has a reputation beyond reputeTom Line has a reputation beyond reputeTom Line has a reputation beyond reputeTom Line has a reputation beyond reputeTom Line has a reputation beyond reputeTom Line has a reputation beyond reputeTom Line has a reputation beyond reputeTom Line has a reputation beyond repute
Re: OPR after Week Seven Events

OPR is an excellent indicator this year.

What OPR does not clarify, however, is autonomous and climbing.

Autonomous is indefensible. I would much much rather have a robot that has an opr of 55 because it scores a 7 disk every single time in auton, then climbs quickly at the end. Even if it has a zero teleop score. Because that's the perfect robot to have play defense.

OPR will never be an end-all measurement, and you really have to understand the mechanics of the game.

Another place that OPR fails is defense. There were several extremely well-scoring machines at MSC that had low OPR's because they spent half their matches playing defense. Unless you scouted and knew that, you'd assume they were just poor robots.

That's the kind of third round sleeper pick that high seeded teams dream of.
Reply With Quote
  #10   Spotlight this post!  
Unread 04-14-2013, 10:25 AM
Karthik's Avatar Unsung FIRST Hero
Karthik Karthik is offline
VEX Robotics GDC Chairman
FRC #1114 (Simbotics)
Team Role: Mentor
 
Join Date: Apr 2002
Rookie Year: 1998
Location: Toronto, Ontario, Canada
Posts: 2,108
Karthik has a reputation beyond reputeKarthik has a reputation beyond reputeKarthik has a reputation beyond reputeKarthik has a reputation beyond reputeKarthik has a reputation beyond reputeKarthik has a reputation beyond reputeKarthik has a reputation beyond reputeKarthik has a reputation beyond reputeKarthik has a reputation beyond reputeKarthik has a reputation beyond reputeKarthik has a reputation beyond repute
Re: OPR after Week Seven Events

Quote:
Originally Posted by PayneTrain View Post
No one's totally sure until Karthik either praises or denigrates it in his strategy presentation this year.
OPR in 2013 as a predictor of a robot's contribution to an alliance = Thumbs Up
OPR in 2013 as a primary or solitary scouting tool = Thumbs Down
OPR in 2013 as a tool to complement effective match scouting = Thumbs Way Up
OPR in 2013 as a stat that is blindly quoted by those without a fundamental understanding of what it means = Thumbs Way Down

For a more detailed analysis, come check out my seminar in St. Louis.
__________________
:: Karthik Kanagasabapathy ::
"Enthusiasm is one of the most powerful engines of success. When you do a thing, do it with all your might. Put your whole soul into it. Stamp it with your own personality. Be active, be energetic, be enthusiastic and faithful and you will accomplish your object. Nothing great was ever achieved without enthusiasm" -- R.W. Emerson
My TEDx Talk - The Subtle Secrets of Success
Full disclosure: I work for IFI and VEX Robotics, and am the Chairman of the VEX Robotics and VEX IQ Game Design Committees
.

Last edited by Karthik : 04-14-2013 at 10:29 AM.
Reply With Quote
  #11   Spotlight this post!  
Unread 04-14-2013, 10:41 AM
IndySam's Avatar
IndySam IndySam is offline
Registered User
FRC #0829 (Digital Goats)
Team Role: Mentor
 
Join Date: Nov 2005
Rookie Year: 2004
Location: Indy
Posts: 3,069
IndySam has a reputation beyond reputeIndySam has a reputation beyond reputeIndySam has a reputation beyond reputeIndySam has a reputation beyond reputeIndySam has a reputation beyond reputeIndySam has a reputation beyond reputeIndySam has a reputation beyond reputeIndySam has a reputation beyond reputeIndySam has a reputation beyond reputeIndySam has a reputation beyond reputeIndySam has a reputation beyond repute
Re: OPR after Week Seven Events

Quote:
Originally Posted by Karthik View Post
OPR in 2013 as a predictor of a robot's contribution to an alliance = Thumbs Up
OPR in 2013 as a primary or solitary scouting tool = Thumbs Down
OPR in 2013 as a tool to complement effective match scouting = Thumbs Way Up
OPR in 2013 as a stat that is blindly quoted by those without a fundamental understanding of what it means = Thumbs Way Down

For a more detailed analysis, come check out my seminar in St. Louis.
How many thumbs do you have?
__________________
"Champions are champions not because they do anything extraordinary but because they do the ordinary things better than anyone else." óChuck Knoll


2012 Boilermaker Engineering Excellence Award
2010 Boilermaker Rockwell Innovation in Control Award.
2009 Buckeye J&J Gracious Professionalism Award
2009 Boilermaker J&J Gracious Professionalism Award
2008 Boilermaker J&J Gracious Professionalism Award
2007 St Louis Regional Winners
Reply With Quote
  #12   Spotlight this post!  
Unread 04-14-2013, 10:51 AM
Andrew Schreiber's Avatar
Andrew Schreiber Andrew Schreiber is offline
Data Nerd
FRC #0125 (NUTRONS) (EWCP)
Team Role: Mentor
 
Join Date: Jan 2005
Rookie Year: 2000
Location: Misplaced Michigander
Posts: 3,207
Andrew Schreiber has a reputation beyond reputeAndrew Schreiber has a reputation beyond reputeAndrew Schreiber has a reputation beyond reputeAndrew Schreiber has a reputation beyond reputeAndrew Schreiber has a reputation beyond reputeAndrew Schreiber has a reputation beyond reputeAndrew Schreiber has a reputation beyond reputeAndrew Schreiber has a reputation beyond reputeAndrew Schreiber has a reputation beyond reputeAndrew Schreiber has a reputation beyond reputeAndrew Schreiber has a reputation beyond repute
Re: OPR after Week Seven Events

Quote:
Originally Posted by IndySam View Post
How many thumbs do you have?
Somewhere, deep in the bowels of 1114's shop, there exists a room full of nothing but students training to be Karthik's hand doubles. They practice thumbs up and thumbs down for hours a day for many years until they are needed in an occasion such as this.
__________________

Planning to top a 4 banner year...
Reply With Quote
  #13   Spotlight this post!  
Unread 04-14-2013, 10:53 AM
Ian Curtis Ian Curtis is offline
Best Available Data
FRC #1778 (Chill Out!)
Team Role: Engineer
 
Join Date: Feb 2005
Rookie Year: 2004
Location: Puget Sound
Posts: 2,434
Ian Curtis has a reputation beyond reputeIan Curtis has a reputation beyond reputeIan Curtis has a reputation beyond reputeIan Curtis has a reputation beyond reputeIan Curtis has a reputation beyond reputeIan Curtis has a reputation beyond reputeIan Curtis has a reputation beyond reputeIan Curtis has a reputation beyond reputeIan Curtis has a reputation beyond reputeIan Curtis has a reputation beyond reputeIan Curtis has a reputation beyond repute
Re: OPR after Week Seven Events

Quote:
Originally Posted by Karthik View Post
OPR in 2013 as a stat that is blindly quoted by those without a fundamental understanding of what it means = Thumbs Way Down
On top of that, it is not terribly difficult to actually match scout and get much more accurate, much more useful data. Fundamentally OPR has way less information to rank teams than a match scouter does, so it should come as a surprise to no one that it doesn't do as good of a job.

Quote:
Originally Posted by Norbert Wiener
The best model of a cat is another, or preferably the same, cat.
Quote:
Originally Posted by George E.P. Box
All models are wrong, some are useful.
__________________
CHILL OUT! | Stability & Control Engineer
Adam Savage's Obsessions (TED Talk) (Part 2)
It is much easier to call someone else a genius than admit to yourself that you are lazy. - Dave Gingery
Reply With Quote
  #14   Spotlight this post!  
Unread 04-14-2013, 10:53 AM
Woolly's Avatar
Woolly Woolly is offline
Missouri State Sophomore (Comp Sci)
AKA: Dillon Woollums
FRC #1806 (S.W.A.T.)
Team Role: Mentor
 
Join Date: Feb 2012
Rookie Year: 2012
Location: Springfield, MO
Posts: 447
Woolly has a reputation beyond reputeWoolly has a reputation beyond reputeWoolly has a reputation beyond reputeWoolly has a reputation beyond reputeWoolly has a reputation beyond reputeWoolly has a reputation beyond reputeWoolly has a reputation beyond reputeWoolly has a reputation beyond reputeWoolly has a reputation beyond reputeWoolly has a reputation beyond reputeWoolly has a reputation beyond repute
Send a message via Skype™ to Woolly
Re: OPR after Week Seven Events

Quote:
Originally Posted by Karthik View Post
OPR in 2013 as a predictor of a robot's contribution to an alliance = Thumbs Up
OPR in 2013 as a primary or solitary scouting tool = Thumbs Down
OPR in 2013 as a tool to complement effective match scouting = Thumbs Way Up
OPR in 2013 as a stat that is blindly quoted by those without a fundamental understanding of what it means = Thumbs Way Down

For a more detailed analysis, come check out my seminar in St. Louis.
In other words, a good strategy would be to use traditional scouting to find robots that complement your robot and strategy, and then look at their calculated contributions along avg. individual scores to help narrow down and rank those robots.
Though at some point, if there is a redundant robot that is a certain amount better than a complimentary robot, you definitely have to consider the redundant robot over the complimentary. How much of a difference must there be to do that? That is a great subject for a team meeting on Friday night.
__________________

Best Finish: 2014- Divisional Semifinalist
Regional Wins: 2013: GKC (1986, 1806, 2457) OKC (1986, 1806, 2389)
Awards: Quality (2014 CIL), Judge's Award (2013 GKC), Innovation in Control (2012 OKC).
2014 Fantasy FIRST Champion (Automated Scoring)
Reply With Quote
  #15   Spotlight this post!  
Unread 04-14-2013, 10:55 AM
Akash Rastogi Akash Rastogi is offline
Currently teamless
no team
Team Role: Alumni
 
Join Date: Feb 2007
Rookie Year: 2006
Location: Mt. Olive, NJ/ Syracuse, NY
Posts: 6,728
Akash Rastogi has a reputation beyond reputeAkash Rastogi has a reputation beyond reputeAkash Rastogi has a reputation beyond reputeAkash Rastogi has a reputation beyond reputeAkash Rastogi has a reputation beyond reputeAkash Rastogi has a reputation beyond reputeAkash Rastogi has a reputation beyond reputeAkash Rastogi has a reputation beyond reputeAkash Rastogi has a reputation beyond reputeAkash Rastogi has a reputation beyond reputeAkash Rastogi has a reputation beyond repute
Re: OPR after Week Seven Events

Quote:
Originally Posted by Karthik View Post
OPR in 2013 as a predictor of a robot's contribution to an alliance = Thumbs Up
OPR in 2013 as a primary or solitary scouting tool = Thumbs Down
OPR in 2013 as a tool to complement effective match scouting = Thumbs Way Up
OPR in 2013 as a stat that is blindly quoted by those without a fundamental understanding of what it means = Thumbs Way Down

For a more detailed analysis, come check out my seminar in St. Louis.
Had to
__________________

Alumni/Mentor FRC 11 - MORT | ['11-'12]: Co-Founder/Mentor FRC 3929 | ['06-'10]: Student FRC 11- MORT | ['08-'12]: Founder - EWCP
Reply With Quote
Reply


Thread Tools
Display Modes Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Forum Jump


All times are GMT -5. The time now is 08:26 AM.

The Chief Delphi Forums are sponsored by Innovation First International, Inc.


Powered by vBulletin® Version 3.6.4
Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Copyright © Chief Delphi