Gee thanks ! It’s an honor to the be named right beside those four teams. But to be fair we didn’t really have a “solid” weekend. We had some intake trouble and we found out that our router was defective. That gave us o lot of issues troughout the weekend. Oh, and we did the lowest score possible of 0 in the final
But our auto was wicked awesome. It failed only once during our second qualification match. But in the finals we had the task of taking ownership of the scale while 1772 did the switch. We did all matches, both opposite and front, flawlessly. When even did 2 cubes in scale in our second semis.
Our climbing mechanism also gave us the opportunity to do a lot of double climb with our 3rd alliance partner 5443. We were able to both climb on the rung without the help of any ramp.
I totally agree with all this. Especially this year. A lot of good scale bot placed low this years. At Montreal, we had 6 rookie team with swith/vault type bot that placed in the top 15. The reasons that this happened is that most scale bot went directly to the scale and ignore the switch. They ended up losing match due to late ownership of their own switch. All the switch bots had to do was getting a cube early in and voila. We didn’t have much defensive play so you kept ownership all the way to the end.
Scouting was extremly important as you could’nt use ranking or OPR as an indication and most of the 15-45 ranked team had the same average cube/match count. Since our first pick (1772) was an highly effective scale bot but could’t climb we went for an OK switch/vault bot(5443) that could climb and that was compatible with our climbing style.
We did have one effective Everybot and it ranked 10th. We had to face them in our quaterfinals since they eventually bacem the 7th alliance captain. I can tell you that those type of bot are annoying!
My concern with OPR this year is essentially similar to Kevin’s, there just not enough sample size. Similar to other games in which teams can contribute meaningfully to the score without obvious quantitative interactions (2009, 2014), there is some inherent value to a metric that seeks to infer the offensive impact of teams beyond the quantity of game pieces they score. However, also similar to 2009 and 2014, so much of the scoring potential of a team depends on their alliance partners and opponents in a given match. I feel there’s simply too much noise in qualification schedule to get a truly accurate OPR measurement.
Instead of OPR, I’d like to see a metric similar to but not exactly the same as CCWM. Pardon my crude math, but imagine a metric that approximately solved a matrix of equations for each match along the lines of MatchPointSpread = Red1 + Red2 + Red3 - Blue1 - Blue2 - Blue3.
Unlike OPR / DPR, the other alliance (and the other alliance’s score) is included, so it would take into account how the strength of the opponents could depress the margin.
As for the actual topic of this thread:
I don’t like using ELO for rankings since it picks up where last year left off (right?) rather than making new numbers each year.
2791 is dope as hell and definitely a top team, pardon my bias
People aren’t mentioning the traditional powerhouses because they don’t need to mention them in order for others to pay attention
I’ve been thinking of making this metric for a long time. (By that I mean several years) But I haven’t had the time to do it myself. However, some statistitions have assured me that it would come out the same as CCWM. I highly encourage you to try it and find out for sure.
To build off of others on this. FUN is 100% on board with having the FRC Top 25 weighted by some sort of objective metric. Really the question is what would be accepted by the majority and how can I have a report ran instantly so it isn’t any more of a hindrance when compiling the FRC Top 25.
I will say that while teams will always slip through the cracks each week that overall this is one of the more accurate community polls IMHO and I believe having around 300 submissions helps with this immensely.
I just wanted to say how much I enjoyed the set up of the region recaps last night! It was a great addition to an already great program. My kids were beyond thrilled to get a mention. Keep up the great work FUN!
Sorry for the long post, but I have a lot of thoughts on below topics.
I’ll be looking in the next few weeks at comparing the predictive power of OPR to Elo for 2018, and compare the predictive power to previous years. People are free to use any metric they like to rank robots, but essentially the only rankings I care about are metrics that are backed by predictive power. My Elo model was purely designed for this purpose. To my knowledge, OPR was not designed with predictive power in mind, but it has turned out to have by far the best predictive power of any common metric, so it is still the gold standard in my mind. That doesn’t mean it isn’t better or worse in some years than others, but I found it’s predictive power was strictly greater than that of the other metrics for every year in 2008-2016. It’s a safe bet it will also be the best metric this year.
I could reset my Elo ratings every year, but nearly everything in my model is done to maximize predictive power for matches, and doing this would severely limit the predictive power of the model, particularly in early weeks of each season. Each team’s Elo in my current model is found by taking 70% of their 2017 end of season Elo and adding 30% of their 2016 end of season Elo, and then reverting this sum by 20% toward 1550 (the pseudo-average Elo). If it were more predictive to revert everyone 100% after each season I would have found this during model tuning. I get that some people might view this as unfair, but my goal is to maximize predictive power, no matter where that takes me, not to make a metric that seems intuitively fair.
We just had a conversation on this very topic here. I didn’t actually go and check for equivalence, but a few years back wgardner did and found the measurements to be distinct. I’m seriously concerned with overfitting with this metric though and it’s predictive power was very poor from my testing, I like EPR a lot better if you want to incorporate who opponents are.
The only metric that would qualify for this privilege is OPR in my opinion. I’ve been working hard to get Elo to be as good or better of a metric than OPR, but considering I’m the only one who calculates it and it doesn’t have nearly as strong of a track record as OPR, it would be a poor choice to use it this year. As I stated above, OPR has consistently been the most predictive metric in FRC. I think it would be cool if the weightings were weighted half by OPR and half by polling. Neither method is anywhere close to perfect, but I think they could complement each other well. I often feel bad when amazing teams with little name recognition attending obscure events don’t make the list, when they are probably better than some teams that did make it.
Coming off the win from MVR, I would definitely agree that 4028 was a robot to contend with. They definitely have a great design, and compared to my team’s, a very similar design haha. I think 379 also has a great robot, with that triple climb working its way into the line-up. 302, being my team’s alliance captain, played smart throughout quals and definitely should be recognized for a great single climb that was fast and sturdy.
Although I haven’t spoke about my team, don’t count us out for coming in strong for our Michigan Events Ohio was just a warm-up.
Overall, had a great time at MVR and I think that a chunk of robots from that regional should not be forgotten.
Thanks for the post. It was highly informative. You make it sound like you have done some serious investigation of the predictive power of different metrics. If that is the case, could you point me in the direction of your findings? I am particularly interested in the relative power of OPR, WMPR, and EPR.
Took a stab at gleaning the top 25 OPR scores for week one (done by hand, so apologies in advance if I’ve missed someone or a typo crept in). Given Caleb’s plug for OPR, I will be interested in seeing how this list compares with the popular polling tonight (noting the hesitation by many to use OPR comparisons across events).
I posted this also in another thread: while the OPR WITHIN an event likely as significant predictive power, the OPRs ACROSS events are likely almost meaningless. This year’s score is highly dependent on the relative strengths of each competition. One regional might be dominated by one bot that can put up only 2 scale cubes, while another might have several that can put up an average of 4 cubes, but they split up the ownership time. Which bots have the highest offensive output? The former likely has a higher OPR.
Not in disagreement with this in general. But I have nevertheless been impressed with the overall effectiveness of bots this early in the season, and would suggest that the dominant bot in your example might only be putting up two scale cubes for strategic reasons, not necessarily due to lack of ability. Despite the limitations of OPR, it will still be interesting to see how many of its top 25 rank in the top 25 of the popular poll.
For whatever it’s worth, 12 of the 25 in the popular poll also had top 25 OPR scores week one (188 and above). The other 13 had OPR scores ranging from 118 to 183, or 161 on average (118 being the outrider). I don’t know whether that would affect anyone’s opinion about the relevance (most thinking irrelevant) of OPR across events.
Ohio was just a warm-up.