I felt like upsets had been way less common so far this year, so I ran some numbers and was surprised to see this stat. Historically over the last 10 years of events (“regular season” events only), the 8th seed is able to advance past their top-seeded opponents at a rate of 9.4%. But such an event has not occurred a single time yet this year.
Some other interesting insights:
The 1 seed has been particularly difficult to beat this year, taking blue banners home in 82.7% of events so far. Regional events have been even more lopsided, where the 1 seed has been winning at a rate of 86.4%. Normally over the last 10 years, the 1 seed only wins 59.8% of events. Here’s a nice little chart:
Playoff upsets across the board have also been much more rare this year. With an “upset” being defined as “any time an alliance advances over a stronger seed at any point in the playoffs”, we’ve been averaging 1.02 upsets per event. Again, this has been worse in regionals specifically where the average number of upsets has been 0.82. Historically this number hovers around 1.85. Here’s another chart:
I’d be interested to hear people’s thoughts on why the brackets have been so chalky lately.
Perhaps events are smaller overall this year, which forces weaker teams into the playoffs who wouldn’t normally be there?
Perhaps the pandemic has taken such a toll on the mid-tier teams that it’s tougher for them to stay competitive with traditional powerhouses?
Perhaps the design of the game this year makes it easier for stronger teams to dominate matches?
Perhaps the ranking system for this game is simply a more accurate reflection of teams’ playoff capabilities than it normally is?
This analysis excludes events like Ontario where the playoffs are reduced to four. I performed it via TBA’s API and some quick scripting - please let me know if I missed something.
That’s my guess. The achievability of the RPs this year, combined with the second sort being average match score, seems to have made the event rankings, at least at the top end, very reflective of the robot’s true ability. I’d attribute the top alliances actually having the best robots to the reason they are usually winning events this year.
This is a really good point and something that applies to both events I’ve attended (28 and 37 teams). The depth of the event drops off quickly enough that there is no way for the lower alliances to get 3 solid scorers that could challenge the top seeds. I wonder if upsets are more common in larger event historically.
I agree that this is probably at least part of it. But it raises an interesting question about the draft as well.
There’s no “right way” to do alliance selection, and there’s no “right” number of upsets to expect in a bracket. But personally, playoffs are a lot less fun for me (both as a player and as a spectator) when the bracket is more likely to be pure chalk than it is to have any upsets at all. The 8th-seed stat is particularly egregious to me - I can’t imagine being at all excited or interested in participating in the playoffs as an 8 seed if it’s a near-certain death sentence. I imagine I’m not the only one who would feel this way.
So if our ranking system is actually improving, does that mean it’s worth re-thinking alliance selection? Does it mean that the draft is actually far more imbalanced than we realized?
I think that this year’s game is also not very well adapted to triple offense. This is where upsets can really happen, as lower seeds can create an advantage due to the snake draft. With the limited number of scoring objects, two offensive bots is probably the cap for a good alliance. And there is rarely a shortage of halfway decent defense bots to go around.
Also, there is plenty of space for a double transversal climb, which the #1 alliance can easily have. This is very different from a game like 2019, where unless a team did some advanced or very creative climb, there was typically only room for one robot on Hab 3.
Yeah I love tracking the playoffs for each event as well and I’ve notched the same stuff it’s crazy! There’s always one or two 8 seeded alliances that win an event each year but not one has cracked the semis even. Some have been very close. Think it could be done at a weaker district or regional if I had to guess. Maybe at a district champ but who knows? It has happened at the championship a few times so you know never! Usually the semis is where the upsets have been mostly like to occur this year.
I’m fairly confident this is the largest contributing factor.
I think the ranking system improving is a good thing. Chalky brackets maybe aren’t the most exciting from a spectator perspective, but that seems like the lesser of two evils when compared with games that penalize the top performing teams via a broken game dynamic.
Ironically double defense proved to be the best strategy for our 7th seeded alliance to make it to Finals.
I actually think triple offense will be a really strong play at champs. The issue at most District/Regional events is that there is a large scoring delta between the top few robots and the rest of the field. 3 mid tier robots can’t keep up with 2 top tier robots in teleop.
I’ll agree with the spectator part of that, for FRC and sports in general. And an 8th seed never having an upset is probably not what I would want as a player either. But is having a dominate 1 seed such a terrible thing? They’ve played well throughout the competition (and generally have a little luck on their side), so shouldn’t they be rewarded? Do we want 8 evenly matched alliances in the playoffs of every event?
The signals from the ranking system are well aligned with the scoring. This largely means first seed is the “best robot” rather than gaming a system.
The smaller event sizes mean more matches. This produces more data for the ranking system.
The decimation of frc teams was not equitable. This largely means the upper tier is likely further from the lower tier of team performance now.
And then yes, the smaller events mean 8/9/10 are a smaller pool.
An interesting data point could be to compare the seeding of alliances across years. My guess is this year it more closely aligns to pick order than usual. Which is probably a signal that the ordering is more correct.
I think there’s a balance to be struck. The historical distribution of event winners prior to this year seemed a lot more ideal - the 1 seed would win 60% of events, the 2 seed would win 17%, the 3 seed would win 9%, the 4 seed would win 6%, etc. I’m wondering why the sudden change in the distribution. For reference, this year the 2 and 3 seeds are winning at half the rate they usually do, and the 4 seed is winning less than 1% of events so far. Some kind of dramatic power shift has occurred.
I will never argue against a more accurate ranking system. It feels cheap to watch a robot abuse bizarre RP mechanics to climb to the top of the rankings, just to show up to the playoffs and get steam-rolled ().
I also agree that chalky brackets are a better alternative to punishing the top performers, but I’d argue that this is a false dichotomy. Game design, event size, etc. are things that can affect event winner distribution without hurting the ranking system.
This gave me a (possibly dumb) idea.
We don’t want the top seed to win every single time, because then if you aren’t one of the top two teams at the event, there’s barely any reason to try (I say this as someone who’s team was on the eighth alliance at one of our events - we tried to win, but it was obvious to everyone we have no chance as soon as our number was called during alliance selection).
On the other hand, if playoffs were a complete tossup, that might be unfair to the teams who worked so so hard to come up on top.
Here’s the idea: randomize the selection order with an uneven distribution, so that the higher the alliance, the more likely they are to choose first. Make it so that the average order across all events is 1-2-…-8.
This way, the higher you are seeded, the better your chances to win, because you will be closer to choosing first. At the same time, the lower alliances might still have a fighting chance because occasionally they would get to choose earlier, thus would have more options to choose from.
There’s a lot of fine-tuning needed here but I think this is an interesting option. What do you think? Good idea, or dumb and overcomplicated?
a better way to do this would be not letting alliance captains pick other alliance captains i think that randomizing alliance selection is a horrible idea, it would suck really hard to lose an event to a die roll
I’ve made this suggestion before, and discovered that it’s unpopular enough that it won’t ever go ahead. But that won’t stop me from trying again
Make matches 5 minutes long (requires new batteries)
Reduce # qual matches to 5 or 6
That will make for more interesting games in my opinion. But will also have the effect of worsening the correlation between “rank” and “goodness”, leading to more surprises during playoffs.
We in FRC have this nerdy fascination with “proportionately rewarding the best”. No other sport goes to such lengths as we do. Most high school sports competitions are like 3 to 5 matches long. Sometimes you get lucky, some times you don’t - and that is part of what keeps it challenging IMO.
I am a Mechanical Engineer [Machine Tool] with GEMS FRC4362. In general, I do not like this Game.
You can win the Cargo RP alone or with very little help. For the Win or Hangar RP, you need decent Alliance partners in Quals. Great partners is even better. Or you need to be one of the best two Robots at the Event.
At Milford District, there was a very small group of good Auton and shooting Robots, as well as a scattering of fairly reliable and decent Robots. There were not enough climbers to have some available for the 1st and/or 2nd Alliance picks.
We had bad Alliance draws and first-Event-self-inflicted wounds. We would not be able to last our way into 1st place. We were not picked by the 1st Alliance, so we were forced to build a solid three Robot Alliance, just like with our 2019 MI State Championship Alliance. We were not the best Robot at either Event. But hopefully we could build the best Alliance. And have that nice 2nd pick partner… So 5 beat 1st and then 3rd. And it was close.
Belleville was a different challenge. There were two great shooters. And there were plenty of climbers. We knew who ever picked 33 Killer Bees likely wins the Event. So we HAD to grab 1st seed and then somehow stay there, as Bees had difficulties during the Event. They would not be in 1st picking.
BUT, we got a much better Alliance draw [especially in our last two Matches] and our Team performed well.
9 Hangar RP and 4 Cargo RP at Milford. 9 Hangar RP and 10 Cargo RP at Belleville.
The mix of the field will point to the obvious strategy to win the Event. But your Alliance draw quality in Quals, Team play quality and the mix of high Scoring, Auton and climbing capable Teams in the Field mix will indicate the likelihood for you to pull off the win. Some holes are just too deep to climb out of without the right Alliance partner skills. This Game relies heavily on a great Quals draw. You gamble a whole lot of design and build time on simply Alliance draw luck…
This year is different than past years where the RP is considered. If you want to rank high, you don’t HAVE to count on having 2 partners that could perform a basic task. I can get the extra RP from hanging if I can have 1 partner get me a mid level climb. I don’t need both. I can get the scoring RP from crushing the goal even if neither partner can. One great robot can do 90% of the work to ensure 2 RP’s even in a loss. This game allows the best teams to overcome weak to dead partners. As long as you have one team with an off the shelf climber we can ensure the extra RP.
Going back, 2020 teams that could hang and had 2 other partners that could hang ranked well. They could even put up a fight in elims because at smaller events the #1 seed might not be able to find that last partner that could hang. You didn’t need to be a great shooter to rank highly.
2019 again the scoring favored balance. If you could do a rocket with defense by yourself you deserve the the extra RP. No guarantee there. Conversely if you could get to the 3rd level, all you needed is a partner to get up to the 1st level to get you the point RP. 1 requires more skill than the other.
2018 seemed a bit more favorable to the top seeds in that a well designed buddy climb and some good strategy could elevate the top teams but again no guarantee that a partner could pull off the buddy strat. A dominant scoring robot could roll through quals.
Please don’t get me started on how unfair 2017 was. You could not get the extra RP if you had a dead partner or a box on wheels.
I could keep going back but those horses are well dead. But what do I know?
Sure it does? FRC is currently a bit like watching a Harlem globetrotters game. Most teams in our league are Washington Generals. It’s no wonder most fold before they turn 5 years old - that’s a long time to wait for any hardware. A little randomness is good for sustainability.
).