Yeah, if you’re looking to see who can actually play the current game better, it would be better to use a different metric. ELO seems more fitting to me to determine the unmeasurable stuff that makes powerhouse teams powerhouses. That is, their ability to win hard matches.
When it comes to quals data, CHS is about average this year
As someone who has competed in FIM, IN, and now CHS, I don’t think average ELO passes a gut check.
IN (anecdotally) has a much different distribution than most FRC regions; there are fewer boxes on wheels and a large group of teams that are all pretty evenly competitive at the top. Makes sense that their average would be higher, but the top end is not as strong as other districts.
What we really need to look at, as has already been mentioned, is distributions. That can tell you a number of things that a single number can’t.
Anecdotally, CHS seems pretty average. Of the districts I’ve competed in, FIM felt stronger, IN was comparable, if not slightly stronger, and NC felt weaker.
I agree with your feeling of FIM being stronger and CHS being average. Maybe the number of rookie has something to do with that. Here is what the rookies look like for 2018:
|Rookies||Rookie ELO||Vet ELO|
|CHS (125 Teams)||5||1448||1517|
|MAR (125 Teams)||9||1436||1540|
|FIM (497 Teams)||64||1438||1508|
|NC (65 Teams)||8||1438||1507|
|PCH (82 Teams)||12||1447||1504|
|IN (49 Teams)||2||1451||1550|
|IS (70 Teams)||8||1453||1524|
|ONT (156 Teams)||25||1440||1523|
|PNW (154 Teams)||7||1447||1518|
|NE (208 Teams)||15||1424||1528|
I guess my theory would be since FIM adds lots of rookies each year, the opportunity to raise the average ELO for the region is reduced compared to districts who add less rookies.
At this point though I’m pretty much just waiting for Caleb to come in here and school me on how I’m using his metrics wrong
You can use them however you want. I’m a bit late to the party, but here are my thoughts.
For comparing across regions, you really have to be careful, because everyone defines the “strength” of a region differently (average? median? top 10%? powerhouse count? etc…) and there’s no objectively correct way to do it. I’d recommend looking at pchild’s region rating graphs here as that gives a fuller picture of each region’s Elo distribution, and you can interpret those however you would like based on your own definition of region strength (or not at all if you don’t like Elo). Here’s another way to break down region strengths by Elo, just plot the Elo percentile versus the Elo rating for each district (done here for FiM, CHS, MAR, and NE):
From this perspective, MAR looks to win out everywhere except the first 10 percentile, probably due in large part to the age of that region. Another way to look is at the Elo rank versus Elo rating, which refers to individual teams:
FiM wrecks the competition on this graph, and my impression is that when people talk about region “strength” they are thinking of something more like this than the first graph. How many super good robots can you name from chs, taking Elo > 1700 that’s like 2. What about NE or MAR? Like 10. What about FIM? Probably like 20 and you know that you’re forgetting a bunch. Especially by the time DCMPs roll around, FIM looks super good with 160 teams at an amazing event. I tend to think more like the first graph though, which means to me that FIM often gets overrated, and are getting a bit of undeserved credit for being competitive when their real strength is being enormous.
Now, regarding the validity of using Elo ratings to compare regions. My Elo ratings work well (or at least better than any other system I’m aware of) at the champs level where there is plenty of mixing between regions. I’ve also twice tried to region balance my Elo ratings, and was unable to get any appreciable predictive power boost out of either of those endeavours. So your conclusion should probably either be that Elo doesn’t have noticeable region bias, or that I’m not clever enough to think of a way to correct for it .
This topic was automatically closed 365 days after the last reply. New replies are no longer allowed.