[FUN] Region Recap Monday - FUNalysis/FRCTop25 Tuesday

FRCTop25 is live https://www.twitch.tv/firstupdatesnow

Are you going to post the results of the top 25+ voting?

Yes :slight_smile: ! We will be posting the Top 40 with ELO this afternoon/evening. I have some delays and am bogged down with real life work so will need to wait until I get home.

1 Like

Sorry on the delay! Here are all the video’s and rankings:

Region Recaps:
Nor’easter Region Week 1 Recap
SEweet Tea Region Week 1 Recap
InFiMidation Region Week 1 Recap
Mouth of the South Region Week 1 Recap
We the North Region Week 1 Recap
Best of the West Region Week 1 Recap
El micrófono Está Encendido Week 2 Preview

FUNalysis:
PNW Mt Vernon SF 4 2019 FUNalysis
ONT Durham College QF3 2019 FUNalysis
FIT Austin Finals 1 2019 FUNalysis

FRCTop25
Ranks 25-11 FRCTop25 Week 1
Ranks 10-6 FRCTop25 Week 1
Ranks 5-1 FRCTop25 Week 1

Week 1 FRCTop25 2019 Rankings **Let us know you think got snubbed or was overrated If you are not in the Top 40 and want to know where you ended up post here or PM me.**

|Team|FRCTop25 rank|ELO Rank|
|118|1|6|
|330|2|3|
|973|3|9|
|33|4|8|
|27|5|4|
|2910|6|1|
|610|7|10|
|1519|8|2|
|3847|9|54|
|2200|10|25|
|3538|11|5|
|5687|12|14|
|1023|13|16|
|2468|14|62|
|4414|15|133|
|88|16|12|
|1718|17|37|
|1807|18|28|
|4481|19|29|
|3309|20|17|
|2714|21|444|
|95|22|11|
|3005|23|21|
|319|23|29|
|624|25|25|
|6329|26|69|
|2539|27|49|
|359|28|18|
|6377|29|75|
|103|30|20|
|3647|31|65|
|4476|32|66|
|4020|33|74|
|188|34|135|
|4454|35|128|
|5803|36|23|
|3990|37|46|
|4564|38|57|
|811|39|149|
|2930|40|27|

FRC Top 10 by Region and taken from the FRCTop25 voting results. Note this is where the team is located from, not the region they played in.

USA Northeast Top 10

1519
5687
88
1807
95
319
6329
2539
103
4454

USA Southeast Top 10

346
4451
1102
2655
1293
1902
6919
86
614
612

FIRST in Michigan Top 10

33
27
3538
1023
1718
3604
3542
548
3641
7491

USA South Top 10

118
3847
2468
2714
3005
624
6377
4020
4063
4610

USA North/Ontario/Quebec Top 10

610
2200
4476
188
3990
1747
5885
7605
379
3986

USA West/British Columbia/Alberta Top 10

330
973
2910
4414
3309
359
3647
5803
2930
1983

International Top 3

4481
5883
5553

Israel and LATAM/South America next week.

1 Like

Sad to see that the number 7 ranked elo team who had the second highest opr this week didn’t make the top 40.

2 Likes
Top 25 Week 1 - ELO Rankings
2910 Jack in the Bot
1519 Mechanical Mayhem
330 The Beach Bots
27 Team RUSH
3538 RoboJackets
118 Robonauts
346 RoboHawks
33 Killer Bees
973 Greybots
610 Crescent Coyotes
95 Grasshoppers
88 TJ(Squared)
1102 M’Aiken Magic
5687 The Outliers
614 Night Hawks
1023 Bedford Express
3309 Friarbots
1747 Harrison Boiler Robotics
359 Hawaiian Kids
103 Cybersonics
3005 RoboChargers
3604 Goon Squad
5803 Apex Robotics
245 Adambots
624 CRyptonite

Source: Sykes Scouting Database 2019

1 Like

346 was better than Robonauts this week

2 Likes

To be fair, the top25 is not really a competitive ranking, rather a popularity contest. Also it can be highly biased by region.

6 Likes

I fully understand it’s just hard to see students put hard work in and it not get recognized.

2 Likes

This comes up every year and there are ways to get people from all over FRC to recognize your team. Just like how ELO puts BBQ in the 400’s it takes consistency and time to be recognized by the over 300 voters who put their input into week 1 voting.

1 Like

I’m sure someone can link this but I know Caleb did a case study that shows that the FRCTop25 is actually pretty accurate. There will always be teams that get left out and deviations but overall it’s pretty darn close.

1 Like

Just for a little bit of background on my Elo ratings, they do not look at single event performances only. Rather each team goes into an event with a seed and their ratings adjust throughout the event based on that seed. Normally the seeds are pretty good and it works out by the end of an event that a team’s rating is a solid reflection of their ability. Sometimes though, there are outlier teams that do dramatically better or worse than their seed. For teams like this, one event will eliminate only roughly half of the bias of their seed. So, if we take 4414 as an example, they started out the event at a rating of 1450 (rookie/new veteran default), and ended the event at a rating of ~1610, for a really dramatic jump of 160 points! However, their actual rating should probably have improved by about 160*2=320, which would put them at 1770, or about 10th in the world. This is indeed borne out from testing, I found 1770 to be the best rating for them in terms of predictive power when I reran Del Mar a few times.

Two groups that are helpful to keep an eye on in addition to the top rated Elo teams are the teams that saw big single event Elo jumps and the highest rated rookies. Both of these groups have lower ratings than they probably should due to their poor seeding going into their event.

Top 5 most improved teams/teams with unexpected performances last week were:
4414
614
4907
4152
346

And top 5 rookies were:
7491
7447
7449
7605
7501

3 Likes

Here’s an excerpt from an email I sent to the FUN team on this topic a few months ago:

I just looked at the pre-champs FRC Top 25 lists for Houston and Detroit. I also generated top 25 lists using teams’ max OPR at this time and current Elo at that time (pre-champs). I chose to use championships District Points (excluding awards) to quantify championships performance. Overall, the “best” list (i.e. the list that was the best predictor of how many champs DP a team would earn) was the FUN list, followed reasonably closely by Elo, and OPR dragged far behind.

Next, I made 4 new lists by taking a weighted average of existing lists. I then found the ordering of how predictive (from most predictive to least predictive) these lists were of champs performance to be:
25% OPR, 75% FUN
25% Elo, 75% FUN
100% FUN
50% OPR, 50% FUN
50% Elo, 50% FUN
100% Elo
100% OPR

To give a sense of magnitude, the 25% OPR, 75% FUN list was about 1 DP better than the 100% FUN list per team, meaning that that the teams on that list on average won half a match more than a team in the same position on the FUN list.

We should really keep in mind this is a pretty small sample size (50 teams) so I’d be wary of drawing too much of a conclusion, but we can pretty clearly say that 100% OPR is the worst of the above lists, and that there’s a good chance a slight adjustment of the FUN list could provide a bit more predictive power of champs performance. The FUN list is certainly one of the better ones though, and we should be wary of adjusting too much.

I’d definitely feel a lot better about drawing sharp conclusions if I had looked at a larger sample size, 50 teams isn’t a lot. It would also be useful to look at lists from earlier weeks and years other than 2018. That said, I do think the predictive power of the FUN lists is comparable to Elo. There are clear biases in the FUN lists (region bias and big-name bias come to mind) but there are also some very obvious biases in Elo (notably veteran team bias, see my above post).

1 Like

Opr also varies in the effective prediction strength depending on the game due to how scoring, ranking and several other factors are not accounted. 2018 may not be a very good year to look at this type of data based on how scoring was possession and time based which does not show how much a team was actually contributing. With games such as 2015 where penalties were negative and scores were linear based on stacks, opr may be a better indicator.

It’s pretty obvious that FRC top 25 is influenced heavily by popularity… Is it “okay”? Sure. Is it “pretty accurate”? Well it certainly doesn’t fit my description.

It’s totally fine for it to be branded as a for fun thing to see who people think are the best teams in FIRST, but branding it as something that is something more than a fun little thing to do to see what people think is pretty proposterous. Citing ELO or OPR comparisons as a legitimization of the rankings is also silly, given the large issues both of those models have with inter-event comparisons.

FRC top 25 is great for a fun comparison between teams and can cause some great discussions about who the top teams actually are, but don’t sell it as a “fact” or call it “accurate”. Doing that just makes you/it look silly.

4 Likes

@Caleb_Sykes Have you ever had a discussion about using a seeding method to come up with a matchmaking algorithm, to either try to balance both sides in a match or to have a NFL type parity schedule where strong teams are matched more often against strong teams.

I follow a lot of Caleb’s work…and he has done exactly that. Check out these 3 TBA blog posts.

I said that a case study showed this and if I misspoke based on Caleb’s response I apologize. Saying I look silly based on you saying: “Well it certainly doesn’t fit my description.” is just saying that your opinion matters more.

View the FRCTop25 however you want, many teams and individuals find value and tune in. The goal has been to bring the FRC community together to celebrate these teams as voted on by community members. While with any poll there will be bias and error, it seems to be that many people like you just completely discount those who vote by assuming they are only voting for teams based on popularity. There are still many people who actually do research and put thought into what they are doing.

Is this poll completely accurate? Of course not, I believe that it tends to be more accurate than what certain people perceive and this is simply a difference of opinion.

2 Likes

This is so interesting that the subjective opinion of the collective is more predictive than going purely by metrics, but when supplemented with metrics only slightly it becomes the best predictor (at least for last year’s world champs). Reminds me of how we typically rank teams during our pick list meetings.

1 Like

One question I have @Caleb_Sykes would be could presence on the FRC top 25 actually improve team performance, as opposed to being a predictor? It’s a morale boost, no doubt. No idea how one would be able to figure this out (but you might).