BTW don't trust game hint threads started by Greg Needel. - ATannahill [more]
 Chief Delphi Breakdown Power Rating (BPR): A New Power Metric
 portal register members calendar search Today's Posts Mark Forums Read FAQ rules

#1
12-05-2017, 01:46 PM
 dc74089 Mentor: Control Systems, Video AKA: Dominic FRC #1902 (Exploding Bacon) Team Role: Mentor Join Date: Dec 2013 Rookie Year: 2011 Location: Orlando Posts: 35
Breakdown Power Rating (BPR): A New Power Metric

A common measure of a team's performance at competition is their OPR (Offensive Power Rating), a metric of how much a team contributes to their alliance's score in an average match. This statistic is limiting: it only discerns between high-performing and low-performing teams. BPR is a new statictic that shows a team's average contribution to a specific scoring target. BPR allows you to look at data beyond a team's estimated contribution to their alliance's score.

A team's BPR in a scoring area shows the team's average contribution to that specific scoring area. For example, a Steamworks robot that scored a large number of points spending the entirety of every match dumping fuel into the low goal would have a high low-goal BPR but a low climbing or rotor BPR. BPR is an entirely objective statistic calculated using data reported by the FMS and the same math as familiar stats like OPR, DPR, and CCWM. There are no human scouters, no subjectivity, and most importantly no delay in reporting BPRs, making it an essential tool for strategy during competition.

For the math nerds: BPR is calculated exactly the same way as with an OPR, DPR, or CCWM calculation: by taking a Cholesky decomposition of a matrix with number of times a team has played in a match with each other team in the event. The only difference is that the decomposition then gets solved with the FMS-reported match breakdown statistics like auto points or teleop fuel low, instead of the total score of the match.

Other FIRST Robotics Competition statistics services typically only show OPR, DPR (Defensive Power Rating), and CCWM (Calculated Contribution to the Winning Margin), and sometimes only show the top 15 teams. The BPR app shows you OPR, DPR, CCWM, and BPR in any numerical statistic reported by the FMS for all teams at any official FIRST regional or championship division.

The BPR app is available to download today on the Google Play store, and a web version will be published to www.frc-bpr.com before kickoff. The app is open source, and the Java library that does all of the heavy lifting to calculate BPR is too.

Happy Scouting!
__________________
-Dominic

FLL - Nerdvana (2011-12)
FTC - Nerdvana/Clockwork Mania 6379 (2012-13), HazMat Robotics 9277 (Student 2014-17, Mentor 2018-?)
FRC - Clockwork Mania 4013 (2014), Exploding Bacon 1902 (Student 2015-17, Mentor 2018-?)

Developer of FTC Self-Inspection, Breakdown Power Rating (BPR), FIRST Alliances, and FRCShirt.trade

“If you never did, you should. These things are fun, and fun is good.” ~Dr. Seuss

Last edited by dc74089 : 12-05-2017 at 04:28 PM. Reason: Change links to hyperlinks
#2
12-05-2017, 01:50 PM
 marcusbernstein I should be at baseball practice FRC #1836 (The Milkenknights) Team Role: Mascot Join Date: Dec 2014 Rookie Year: 2013 Location: Philadelphia Posts: 482
Re: Breakdown Power Rating (BPR): A New Power Metric

Sounds very cool! How does this compare to Component OPR? Is it the same thing? Did I totally misunderstand something?

Either way, seems like a useful tool so thanks for sharing!
__________________

"Once you've concluded that it isn't a code problem, it isn't an electrical problem, and it isn't a mechanical problem, after all that blaming you now have a culture problem."
"Precedence is no excuse for being wrong."
"Aesthetics are great, but not at the expense of functionality."
"An entire team rallied around a potentially wrong decision is better than half a team rallied around the right one. And if that potentially wrong decision ends up being the right one, that unified team will be successful."
#3
12-05-2017, 01:52 PM
 AllenGregoryIV Engineering Coach AKA: Allen "JAG" Gregory FRC #3847 (Spectrum) Team Role: Coach Join Date: Jul 2008 Rookie Year: 2003 Location: Texas Posts: 3,086
Re: Breakdown Power Rating (BPR): A New Power Metric

Thanks for making this. Anyway you can add off-seasons that have the FMS match breakdowns.
__________________

Team 647 | Cyber Wolf Corps | Alumni | 2003-2006 | Shoemaker HS
Team 2587 | DiscoBots | Mentor | 2008-2011 | Rice University / Houston Food Bank
Team 3847 | Spectrum | Coach | 2012-20... | St Agnes Academy
LRI | Alamo Regional | 2014-20...
"Competition has been shown to be useful up to a certain point and no further, but cooperation, which is the thing we must strive for today, begins where competition leaves off." - Franklin D. Roosevelt
#4
12-05-2017, 01:54 PM
 noah.gleason Registered User FRC #0449 (The Blair Robot Project) Team Role: Programmer Join Date: Mar 2017 Rookie Year: 2016 Location: Bethesda, MD Posts: 139
Re: Breakdown Power Rating (BPR): A New Power Metric

Looks like a cool new tool! Have you run any tests on this? E.g., make a simulated regional/district event/etc where you know how well each team can do each task (e.g. each delivers a normally distributed number of gears per match, you know the mean) and see how well what BPR gives you matches up with the values you know.
#5
12-05-2017, 01:55 PM
 dc74089 Mentor: Control Systems, Video AKA: Dominic FRC #1902 (Exploding Bacon) Team Role: Mentor Join Date: Dec 2013 Rookie Year: 2011 Location: Orlando Posts: 35
Re: Breakdown Power Rating (BPR): A New Power Metric

Quote:
 Originally Posted by marcusbernstein Sounds very cool! How does this compare to Component OPR? Is it the same thing? Did I totally misunderstand something?
I had done a few searches to see if anything like this existed yet, but I didn't come across Component OPR. From a glance, it looks to be the same calculation, but in a Java/Android wrapper instead of Excel
__________________
-Dominic

FLL - Nerdvana (2011-12)
FTC - Nerdvana/Clockwork Mania 6379 (2012-13), HazMat Robotics 9277 (Student 2014-17, Mentor 2018-?)
FRC - Clockwork Mania 4013 (2014), Exploding Bacon 1902 (Student 2015-17, Mentor 2018-?)

Developer of FTC Self-Inspection, Breakdown Power Rating (BPR), FIRST Alliances, and FRCShirt.trade

“If you never did, you should. These things are fun, and fun is good.” ~Dr. Seuss
#6
12-05-2017, 01:58 PM
 dc74089 Mentor: Control Systems, Video AKA: Dominic FRC #1902 (Exploding Bacon) Team Role: Mentor Join Date: Dec 2013 Rookie Year: 2011 Location: Orlando Posts: 35
Re: Breakdown Power Rating (BPR): A New Power Metric

Quote:
 Originally Posted by noah.gleason Looks like a cool new tool! Have you run any tests on this? E.g., make a simulated regional/district event/etc where you know how well each team can do each task (e.g. each delivers a normally distributed number of gears per match, you know the mean) and see how well what BPR gives you matches up with the values you know.
The most in-depth "test" I ran was comparing my team's rankings to what I remember of our performance over the past 3 seasons, nothing beyond "I remember us being great at this in this event", "we were eh at this" and "we never did this on the field (ahem fuel)", and the numbers seemed to make sense with my recollections. :-P
__________________
-Dominic

FLL - Nerdvana (2011-12)
FTC - Nerdvana/Clockwork Mania 6379 (2012-13), HazMat Robotics 9277 (Student 2014-17, Mentor 2018-?)
FRC - Clockwork Mania 4013 (2014), Exploding Bacon 1902 (Student 2015-17, Mentor 2018-?)

Developer of FTC Self-Inspection, Breakdown Power Rating (BPR), FIRST Alliances, and FRCShirt.trade

“If you never did, you should. These things are fun, and fun is good.” ~Dr. Seuss

Last edited by dc74089 : 12-05-2017 at 02:01 PM.
#7
12-05-2017, 02:15 PM
 Caleb Sykes Knock-off Dr. Strange AKA: inkling16 no team (The Piztons) Join Date: Feb 2011 Rookie Year: 2009 Location: Minneapolis, Minnesota Posts: 1,771
Re: Breakdown Power Rating (BPR): A New Power Metric

Quote:
 Originally Posted by dc74089 The most in-depth "test" I ran was comparing my team's rankings to what I remember of our performance over the past 3 seasons, nothing beyond "I remember us being great at this in this event", "we were eh at this" and "we never did this on the field (ahem fuel)", and the numbers seemed to make sense with my recollections. :-P
I checked most of your fields against the 4536 scouting database's, and the ones I could get data for matched. Some of the fields though gave me an error saying "Please select a numeric field." A sample of these were:
robot1Auto
rotor4Engaged
rotor1Auto

These are all fields which contain boolean values when requested from FIRST's or TBA's API, so you probably need to add in a conversion to 0 or 1 for these fields before you begin calculations.

Looks pretty cool though!
#8
12-05-2017, 02:28 PM
 dc74089 Mentor: Control Systems, Video AKA: Dominic FRC #1902 (Exploding Bacon) Team Role: Mentor Join Date: Dec 2013 Rookie Year: 2011 Location: Orlando Posts: 35
Re: Breakdown Power Rating (BPR): A New Power Metric

Quote:
 Originally Posted by Caleb Sykes These are all fields which contain boolean values when requested from FIRST's or TBA's API, so you probably need to add in a conversion to 0 or 1 for these fields before you begin calculations.
Yeah, I haven't written the code to deal with booleans because the data they return can get kind of weird. For example, the touch plates are split into three booleans, one per touch plate. That doesn't refer to the team in position 1, 2 or 3 but the actual physical touch plate location, so unless every team climbs using the same position every match, the statistic isn't meaningful. Rotors is interesting though, because it is shared between all teams.

The best way to get a meaningful number out of those scoring targets is to use autoMobilityPoints, teleopRotorPoints, and autoRotorPoints respectively. If you want to play around with doing things differently, the Java library I linked has a really easy way to use a custom metric. Look at getForSupplier and MetricProvider.
__________________
-Dominic

FLL - Nerdvana (2011-12)
FTC - Nerdvana/Clockwork Mania 6379 (2012-13), HazMat Robotics 9277 (Student 2014-17, Mentor 2018-?)
FRC - Clockwork Mania 4013 (2014), Exploding Bacon 1902 (Student 2015-17, Mentor 2018-?)

Developer of FTC Self-Inspection, Breakdown Power Rating (BPR), FIRST Alliances, and FRCShirt.trade

“If you never did, you should. These things are fun, and fun is good.” ~Dr. Seuss
#9
12-05-2017, 02:29 PM
 JesseK Expert Flybot Crasher FRC #1885 (ILITE) Team Role: Mentor Join Date: Mar 2007 Rookie Year: 2005 Location: Reston, VA Posts: 4,229
Re: Breakdown Power Rating (BPR): A New Power Metric

There's a COPR breakdown in Java, just check my signature.

It was useful for understanding robot capabilities at DCMP, WCMP and IRI. Based on it, I could judge whether it was worth the risk to go for 4 rotors. Yet it was not a good predictor of whether the full potential of the robot would be met in a given match.
__________________
Drive Coach, 1885 (2007-present)

#10
12-05-2017, 02:41 PM
 dc74089 Mentor: Control Systems, Video AKA: Dominic FRC #1902 (Exploding Bacon) Team Role: Mentor Join Date: Dec 2013 Rookie Year: 2011 Location: Orlando Posts: 35
Re: Breakdown Power Rating (BPR): A New Power Metric

Quote:
 Originally Posted by JesseK There's a COPR breakdown in Java, just check my signature. It was useful for understanding robot capabilities at DCMP, WCMP and IRI. Based on it, I could judge whether it was worth the risk to go for 4 rotors. Yet it was not a good predictor of whether the full potential of the robot would be met in a given match.
That's a fork of my frc-power-calc repository that I linked at the bottom of the first post! This isn't a new project, it's been around and public for about a year, I just never posted it anywhere so I assumed nobody had found it yet. I just spent a week or two updating it to the new V3 TBA API, made a huge performance fix, refactored some stuff, and wrapped it all in an Android app.

I'm so glad somebody was able to find a use for the library during competition! It looks like you've made some great modifications to it. If there's ever anything you'd like to contribute, send over a pull request!
__________________
-Dominic

FLL - Nerdvana (2011-12)
FTC - Nerdvana/Clockwork Mania 6379 (2012-13), HazMat Robotics 9277 (Student 2014-17, Mentor 2018-?)
FRC - Clockwork Mania 4013 (2014), Exploding Bacon 1902 (Student 2015-17, Mentor 2018-?)

Developer of FTC Self-Inspection, Breakdown Power Rating (BPR), FIRST Alliances, and FRCShirt.trade

“If you never did, you should. These things are fun, and fun is good.” ~Dr. Seuss

Last edited by dc74089 : 12-05-2017 at 02:45 PM.
#11
12-05-2017, 03:03 PM
 JesseK Expert Flybot Crasher FRC #1885 (ILITE) Team Role: Mentor Join Date: Mar 2007 Rookie Year: 2005 Location: Reston, VA Posts: 4,229
Re: Breakdown Power Rating (BPR): A New Power Metric

Quote:
 Originally Posted by dc74089 That's a fork of my frc-power-calc repository that I linked at the bottom of the first post! This isn't a new project, it's been around and public for about a year, I just never posted it anywhere so I assumed nobody had found it yet. I just spent a week or two updating it to the new V3 TBA API, made a huge performance fix, refactored some stuff, and wrapped it all in an Android app. I'm so glad somebody was able to find a use for the library during competition! It looks like you've made some great modifications to it. If there's ever anything you'd like to contribute, send over a pull request!
Ah, I see! Nice to meet you!

I'm not sure I kept anything general enough to make a contribution back, but I will definitely look into it.
__________________
Drive Coach, 1885 (2007-present)

#12
12-05-2017, 03:12 PM
 Doug Frisk Keeping Score AKA: Doug Frisk no team Join Date: Mar 2014 Rookie Year: 2011 Location: Behind the FMS Posts: 566
Re: Breakdown Power Rating (BPR): A New Power Metric

Quote:
 Originally Posted by dc74089 Yeah, I haven't written the code to deal with booleans because the data they return can get kind of weird. For example, the touch plates are split into three booleans, one per touch plate. That doesn't refer to the team in position 1, 2 or 3 but the actual physical touch plate location, so unless every team climbs using the same position every match, the statistic isn't meaningful. Rotors is interesting though, because it is shared between all teams. The best way to get a meaningful number out of those scoring targets is to use autoMobilityPoints, teleopRotorPoints, and autoRotorPoints respectively. If you want to play around with doing things differently, the Java library I linked has a really easy way to use a custom metric. Look at getForSupplier and MetricProvider.
When analyzing, I made the assumption that the touchpads correspond to the driver's station in front of them. Red 1 = red touchpad near, red 2 = red touchpad middle, red 3 = red touchpad far. Blue 1 = blue touchpad far, Blue 2 = blue middle and Blue 3 = blue near.

It isn't the perfect alignment, but I'd estimate that 90% of the time, that corresponds to where a team climbed.
__________________
To Do list:
Orange County Regional
Northern Lights Regional
Great Northern Regional
Central Illinois Regional
Minnesota North Star Regional
Seven Rivers Regional
#13
12-05-2017, 03:36 PM
 Basel A It's pronounced Basl with a soft s AKA: @BaselThe2nd FRC #3504 (Girls of Steel) Team Role: College Student Join Date: Mar 2009 Rookie Year: 2009 Location: Pittsburgh, PA Posts: 2,294
Re: Breakdown Power Rating (BPR): A New Power Metric

Thanks for posting this! There's been a lot of discussion and work on component OPR, but I believe this is the first project that puts it online or in an app, calculated live. That's very helpful!
__________________
UMich '17, Carnegie Mellon '18
Team 2337 | 2009-2012 | Student
Team 3322 | 2014-2017 | College Student
Team 3504 | 2018-Present | Grad Student / Engineer
#14
12-05-2017, 03:55 PM
 dc74089 Mentor: Control Systems, Video AKA: Dominic FRC #1902 (Exploding Bacon) Team Role: Mentor Join Date: Dec 2013 Rookie Year: 2011 Location: Orlando Posts: 35
Re: Breakdown Power Rating (BPR): A New Power Metric

Quote:
 Originally Posted by Doug Frisk It isn't the perfect alignment, but I'd estimate that 90% of the time, that corresponds to where a team climbed.
None of this is perfect, unfortunately. To get perfect data you'd need a human scouting each match. If we assume that each robot climbs in front of their driver station, we don't need to do any of the fancy linear algebra with the BPR calculation, we can just take the average of how many times we think a team climbed (how many times the touchpad in front of their DS was triggered) over how many matches they played. I'm curious to see how the effect of the uncertainty over where a robot climbs compares with the mathematical guesstimation that comes with the linear algebra, and how both of those compare to human scouting data.

But alas, it's finals week, and I've already spent too much time choosing robotics over school...
__________________
-Dominic

FLL - Nerdvana (2011-12)
FTC - Nerdvana/Clockwork Mania 6379 (2012-13), HazMat Robotics 9277 (Student 2014-17, Mentor 2018-?)
FRC - Clockwork Mania 4013 (2014), Exploding Bacon 1902 (Student 2015-17, Mentor 2018-?)

Developer of FTC Self-Inspection, Breakdown Power Rating (BPR), FIRST Alliances, and FRCShirt.trade

“If you never did, you should. These things are fun, and fun is good.” ~Dr. Seuss
#15
12-05-2017, 04:40 PM
 dc74089 Mentor: Control Systems, Video AKA: Dominic FRC #1902 (Exploding Bacon) Team Role: Mentor Join Date: Dec 2013 Rookie Year: 2011 Location: Orlando Posts: 35
Re: Breakdown Power Rating (BPR): A New Power Metric

Quote:
 Originally Posted by AllenGregoryIV Thanks for making this. Anyway you can add off-seasons that have the FMS match breakdowns.
Sorry I missed your post! There's currently a bug in The Blue Alliance's API that is preventing me from easily getting offseason event data. Instead of putting time into a workaround, I am going to try to wait for them/work with them to fix it (and they're usually pretty quick at fixing these type of things).
__________________
-Dominic

FLL - Nerdvana (2011-12)
FTC - Nerdvana/Clockwork Mania 6379 (2012-13), HazMat Robotics 9277 (Student 2014-17, Mentor 2018-?)
FRC - Clockwork Mania 4013 (2014), Exploding Bacon 1902 (Student 2015-17, Mentor 2018-?)

Developer of FTC Self-Inspection, Breakdown Power Rating (BPR), FIRST Alliances, and FRCShirt.trade

“If you never did, you should. These things are fun, and fun is good.” ~Dr. Seuss

 Thread Tools Display Modes Rate This Thread Linear Mode Rate This Thread: 5 : Excellent 4 : Good 3 : Average 2 : Bad 1 : Terrible

 Posting Rules You may not post new threads You may not post replies You may not post attachments You may not edit your posts vB code is On Smilies are On [IMG] code is Off HTML code is Off
 Forum Jump User Control Panel Private Messages Subscriptions Who's Online Search Forums Forums Home Announcements     User Announcements FIRST     General Forum     Rumor Mill     Career     Robot Showcase Technical     Technical Discussion     Robotics Education and Curriculum     Motors     Electrical         CAN     Programming         NI LabVIEW         C/C++         Java         Python     Sensors     Control System     Pneumatics     Kit & Additional Hardware     CAD         Inventor         SolidWorks         Creo     IT / Communications         3D Animation and Competition         Website Design/Showcase         Videography and Photography         Computer Graphics Competition     Awards         Chairman's Award     Rules/Strategy         Scouting     Team Organization         Fundraising         Starting New Teams         Finding A Team         College Teams     Championship Event     Regional Competitions     District Events     Off-Season Events     Thanks and/or Congrats     FRC Game Design     OCCRA         OCCRA Q&A Other     Chit-Chat         Games/Trivia         Fantasy FIRST     Car Nack's Corner     College & University Education     Dean Kamen's Inventions     FIRST-related Organizations         The Blue Alliance     FIRST In the News...     FIRST Lego League     FIRST Tech Challenge     VEX     Televised Robotics     Math and Science     Robot in 3 Days (RI3D)     NASA Discussion ChiefDelphi.com Website     CD Forum Support     Extra Discussion

All times are GMT -5. The time now is 03:50 AM.

The Chief Delphi Forums are sponsored by Innovation First International, Inc.

 -- English (12 hour) -- English (24 hour) Contact Us - Chief Delphi - Rules - Archive - Top