Re: Strongest regional competitions
 

Don't look just at the top 5, look at the top 25. Also, does anyone else remember the fiasco that was Texas A&M beating South Carolina and rocketing up to top 5? That's what I mean by how these rankings are unnecessary.


Back to the point at hand:


I agree with this. Part of what we do in Fantasy FIRST is try to predict who will do well on the field and with chairmans. Sometimes unpredictable things can happen, but for the most part things in FRC are pretty stable.

Re: Strongest regional competitions
 

The hardest/strongest regional(s) is (are) always the one(s) you attend. Remember, you can only really understand how good a team is if you see them live! :)

Re: Strongest regional competitions
 

But only 26 teams at Waterloo made eliminations last year and I'd suspect many of those made it at Waterloo, which likely was one of the easier events to get picked at. :)

Re: Strongest regional competitions
 

I finally got around to the tedious task (for me, without scripting capabilities) of putting all the current team lists for the events into Excel and looking up aWAR for each team.

If you're new to aWAR, it is essentially a year-by-year statistic to describe how good/successful a team was in a certain year. It is primarily based on the district point system (i.e., teams accrue points based on how they do in quals, picking, elims, and awards), plus an addition from OPR. District and CMP points have a 1.7x multiplier. This raw point value is then adjusted and scaled so that a 'replacement level' team (I determined a replacement level team to be one which is on the cusp of making elims at a 40-team district event, with a 5-7-0 record who may or may not win some awards) has an aWAR around 0 and a dominant team that year should have an aWAR around 7. aWAR stands for 'aggregate Wins Above Replacement,' explaining why it is scaled as such... an aWAR of 0 would suggest a team capable of being on the cusp of making elims, an aWAR of 7 would suggest a team capable of getting a perfect record in quals. For more information about aWAR, see this thread where I first introduced it (http://www.chiefdelphi.com/forums/sh...8&postcount=1).

At any rate, for this ranking I used teams' multi-season aWAR... the weighted average of the last 4 years (40%, 32%, 20%, 8%). For each event I took the average of all teams, the average of the top 8 teams by aWAR, the average of the top 16 teams by aWAR, and the average of the top 24 teams by aWAR. Each are given below.

The average aWAR could theoretically evaluate how good qualification matches, whereas the average of the top 8 may be better for determining which events have the most 'star power', and the average of the top 24 may be better for evaluating how competitive elims will be. Note that the averages of the upper subsets (particularly the top 24) will favor large events and the average of all teams may be pretty foreign from how folks usually evaluate how good a tournament is (looking at the top few teams).

As is starting to be discussed some in this thread, no statistical description of teams performance (whether over the team's history, recent years, or last year) will perfectly describe the team's success... however, you see teams generally performing around similar levels (i.e., perennial powerhouses, perennial contenders, teams that sometimes build contenders, teams that have never made elims, etc.). The idea is that you can predict roughly how competitive events will be relative to others... obviously this can only go so far.

I'm attaching a spreadsheet that allows you to go through and sort/rank the events yourself (perhaps filter out categories?) and also allows you to sort/rank teams by aWAR... I recommend this particularly for those of you that are intrigued by or skeptical of aWAR. Filter down to teams from your area and see what you think... I hope to upload my full aWAR spreadsheet that allows you to use the lookup feature to 'study' teams' recent years and individual seasons (similar to the lookup sheet on Ed Law's spreadsheet).

As a last note, almost all the groundwork to calculate aWAR was laid by Dan Niemitalo (Nemo), who made the spreadsheet for the same reason but to generate a statistic he called Performance Index.



Sorted by Top 8 aWAR:

Sorted by Top 16 aWAR:

Sorted by Top 24:

Sorted by Average aWAR:

Re: Strongest regional competitions
 

Before anybody allows themselves to read the following and experience angst over the obvious shortcomings of using imperfect past metrics to evaluate a team's performance, remember the following: this is just for fun.

I've been toying with concepts like this in my head as well. I'd like to create an index that tries to compare performance at different events more fairly, similar to the way baseball stats can adjust for things like park effects, league effects, different levels of offense in different years, and so on.

Rather than average OPR for the event, for evaluating replacement level I'd probably favor something like average OPR of teams 20-28 on the OPR list. And for evaluating the difficulty of winning the event, I'd probably look at the average OPR of the top 4-8 teams, or maybe even just the top 2-3.

It depends what one is looking for. If you want to know how hard an event is to WIN, then you mainly need to look at the strength of the top two teams other than your own team to know how high the bar is. If you want to gauge how hard it is to make the semifinals, on the other hand, you're probably looking at the strength of the top 10 or so teams, because you want to be in that group to have a good shot at getting on the top 5 or so alliances and avoid being the underdog in the quarterfinals.

I'm just spitballing here, but I think it might make sense to weight the value of an event win by average of the top ~3 teams, compared to the average of the top ~3 across ALL events. Then assign more or less value to a win based on how the event stacks up. And that's for 1st and 2nd robots on the alliance - I'd probably want to do something different for the 2nd pick or a backup robot. For finalists (robots 1+2) I'm probably looking at the average of robots ~3-5 compared to that average for all events. And so on. This has issues and it's a loose idea in my mind so far, but I think that would provide a bit more of a basis for comparing a team's win at Event A to another team's semifinalist finish at Event B.