836 - The RoboBees, amazing team and outreach, and always in contention for Chairman’s!!
And of course 1629!
So I have this script I’ve been using for the past year now that calculates CPR (Chairman’s Power Rating) values for teams at varying events. It’s been pretty accurate for regionals (multiple 2016 regionals had 100% prediction accuracy), so I’ll post what it’s predictions are for North and South Champs. For Champs, it generates a list of a couple “possible candidates” and then vaguely ranks them. All CCA winners from the past 10 years (that’s as far as I’ve checked presently) have been on the possible candidate list my script generates. Note that rankings go off of the team’s award history and can’t take into account the “quality” of the team’s outreaches and Chairman’s submissions, and the teams processed by the script were all of the 2016 Chairman’s winners that attended Champs.
North (St. Louis):
1. 1241 (THEORY6) - 4.91 CPR
2. 1305 (Ice Cubed) - 4.5 CPR
3. 1710 (The Ravonics Revolution) - 4.33 CPR
4. 1885 (ILITE Robotics) - 4.0 CPR
5. 503 (Frog Force) - 4.0 CPR
6. 1511 (Rolling Thunder) - 3.92 CPR
7. 2614 (Mountaineer Area RoboticS (MARS)) - 3.83 CPR
8. 384 (Sparky 384) - 3.83 CPR
9. 537 (Charger Robotics) - 3.5 CPR
10. 1086 (Blue Cheese) - 3.25 CPR
11. 11 (MORT) - 1.5 CPR
South (Houston):
1. 2468 (Team Appreciate) - 4.33 CPR
2. 604 (Quixilver) - 4.0 CPR
3. 2486 (CocoNuts) - 4.0 CPR
4. 1311 (Kell Robotics) - 4.0 CPR
5. 1540 (Flaming Chickens) - 4.0 CPR
6. 1902 (Exploding Bacon) - 3.67 CPR
7. 3132 (Thunder Down Under) - 2.74 CPR
8. 812 (The Midnight Mechanics) - 2.58 CPR
9. 233 (The Pink Team) - 2.58 CPR
10. 932 (Circuit Chargers) - 2.25 CPR
11. 1868 (Space Cookies) - 2.0 CPR
Interesting numbers, thank you for sharing! It’s interesting to compare and contrast this list with the more qualitative evaluations shared on this thread. I’m a bit of a stat geek and have wondered about quantifying Chairman’s chances for a while now (just for fun, of course), so I have a few questions:
- How is CPR calculated? Is it based exclusively on Chairman’s or are other awards and factors considered?
- Exactly how accurate is it in predicting Regional/District/DCMP Chairman’s Awards?
- How does it do with predicting historic Championship Chairman’s Award winners?
- How much of an improvement is CPR over more naive metrics, such as total previous Chairman’s Awards?
- How well does it anticipate “outsiders” who have never won Chairman’s receiving the award at the Regional/District level?
I’d love to tinker with your algorithm/code if you’d be willing to share.
-
To quickly summarize, CPR is calculated based off of Chairman’s and Engineering Inspiration wins in the past four years (inclusive of the current one). Chairmans and E.I. both have base point values (currently 1 and .33) that are reduced then added to a team’s CPR based off of when that award was won (i.e. winning Chairman’s in 2016 is 1 point, winning Chairman’s in 2015 is 0.75 points, 2014 is .5, ect…). There’s also a point bonus for winning Chairman’s at an event with another “high” CPR team present. I’ve experimented with also awarding points for streaks (winning multiple Chairman’s and/or E.I.'s in a row), but I’ve found it tends to cause fluctuations in accuracy. I’m always tweaking the point values for everything and attempting to add new ways of increasing accuracy, but this is currently the most accurate combination of factors I’ve gotten.
-
For regionals and district championships, the script is fairly accurate at guessing the winners/likely winners. It guessed the correct Chairman’s and E.I. winners (1st on the rankings being the Chairman’s winner, 2nd often being E.I.) for numerous events and when it was wrong, was within a rank or two of being 100% correct (i.e. rank 3 winning Chairman’s). It tends to be more inaccurate at guessing district events due to the larger amount of “upsets” and sometimes lack of any teams with a Chairman’s background. As a typical rule, the more Chairman’s history that exists for the teams at an event, the more accurate the predictions will be.
-
Champs predictions are more difficult because at champs, the winner is chosen based off of the quality of their submissions and outreach, which my script doesn’t (and can’t) consider when computing CPR. However, the script is very good at supplying a small list ranging 10-20 teams that “might” win the award. For at least the past 10 years, the HoF winner has been on the list the script provide, and is usually in the top 10 at the very least. For example, 987 was 8th for 2016’s predictions, 597 was 10th for 2015’s, 27 was 4th for 2014’s, and 1538 was 3rd for 2013’s.
-
I would say CPR is quite a bit better than just counting Chairman’s awards. The script actually started off initially as a way to quickly display how many Chairman’s awards the teams at an event had won, and the “predictions” it helped generate then were a lot more inaccurate then what CPR gives now.
-
This is probably the biggest weakness of the current script. All of it’s stats are in some way based off of the team’s Chairman’s and E.I. history, so it never expects a team with a very weak (or non-existent) Chairman’s record to win.
I’ll be making the code public soon. I’m actually in the process of re-writing most of it to be user-friendly and readable by another developer. If you have any more questions about it (or want to see predictions for certain event(s) in the past or future) just let me know 
I may start posting CPR predictions for 2017 regionals once they start getting closer, though I’ve already been running some internally.
Some great summary reads today!
How could I forget about 1311, 604, 1540, and 2486? So many great teams in the running for both championships…
I know you may not have meant to, but this implies that Chairman’s at other events is not chosen based on the quality of their submission.
I know at regionals and districts the judges may know the teams more personally, but I feel like the award is still given based upon the submission and outreach quality, right?
I was a little confused by this statement, too. The Chairman’s Award is based on the submission at all levels. While it is easy to assume that a team with a streak or a lot of relevant wins would be the “best”, Judges are trained to do their best to objectively decide who will earn each award based on their submissions.
You guys are taking part of fast response to questions about my script waaay out of it’s context and intended meaning. I was referring to how there’s usually a gap in the teams at regionals and that makes it easier to predict wins for the script. At champs the teams, statistically, are tied and the script doesn’t know who to predict, since the deciding factor (submission quality) isn’t considered by the script. I was in no way implying regional Chairman’s (or any Chairman’s for that matter) wasn’t quality or submission based. Hate to cause confusion, would never disrespect the award or the people judging it
For north champs I have to think 503 is close to a lock…they’re way overdue.
South champs will be a bit more competitive, in my mind. I’m biased towards 1540, of course, and between 1311 (under the radar but an amazing program), 2486 (running FIRST in Arizona, basically), and 3132 (might be a bit young still, but who else can say they started FIRST on a continent?), Houston should be really competitive.
I would have to agree with Jacobs predictions. Those four team are all very deserving of the award and more than qualified to win it IMO. I would personally love to see 1540 win it since I have seen the work that they do first hand here in Oregon. Alternatively I think that 1311 has a great shot too, having talked with them before their program and commitment to changing culture for the better is unprecedented.
You guys and 836 are definitely strong contenders for CHS DCMPs, but I don’t know about worlds. I mean, what teams have won chairmans at worlds following not winning it at either their DCMP or a regional the prior year? (as in got to present chairmans at worlds the prior year)
I am 90% sure the first CHS team to win chairmans at champs will be 1885. If that is meant to happen this year, I don’t know, but I don’t think anyone other than 1885 is basically a lock to win Chairmans at DCMPs, and therefore the only extremely strong contender for worlds. Also 1885 is fourth for CPR at north champs.
In case you didn’t know, 1629 won Engineering Inspiration at Worlds in 2011 and 2012. And that was before it was split up into divisions. They definitely have the program to make it happen.
Chesapeake is a pretty strong region for that award. I agree with you: my money would also be on 1885. But don’t sleep on GaCo, 836, and the lovable cheeseheads (1086). 384 also has a propensity for winning the award in the region. They could surprise.
I was aware they won it at worlds, but didn’t know that was before they split it up into divisions. Impressive. I did not mean to sound like I was talking bad about 836 or 1629, they are actually two of my personal favorite teams in CHS.
2537 started one, almost two FRC teams in 2016. We wouldn’t have more than an outside chance of beating either of them in a hypothetical chairman’s head to head.
I also am aware of all of the extremely impressive things 1086 and 384 did to earn a spot over the likes of 836 and 1629 and other impressive awards teams.
But I still take 1885 on the (half) world stage any day of the week.
In my experience as a submitter, it isn’t. There has been feedback given that implied that “you guys have won enough” which is explicitly NOT in the criteria.
Having won a few times and submitted a couple more, I have had a few similar experiences. That’s one of the reasons I was asking.
But how Chairman’s is judged and decided is a completely different discussion.