The inspiration for this article came from a similar 538 article a few years back on the Patriots. The first thing we might want to ask is, “How many years does a team need to do well to be considered a dynasty?” Should we just look at 4-year periods since that’s the max years a student can be on a team? Or should we just look at single year periods since every game is so different? A main idea from the 538 article is that it’s kind of silly to ask such questions, as a true dynasty should clearly stand above the field no matter how you chop up the data. I was originally planning to use a very similar visualization method as theirs, more in line with what I made when talking about dynasties back in this thread. However, there are a pair of very notable differences between FRC dynasties and NFL dynasties which I will expand on below. Like a lot of my projects, I’ll be using my Elo ratings as a baseline “team strength” statistic, because it’s very flexible and it’s the best general predictor of competitive success that I’m aware of.
I’m not sure, I didn’t manage that site so I couldn’t tell you (although I believe it used a variant of my Elo model). I’ll have week-by-week Elos in my scouting database, and you can also get live Elos out of my event simulator.
Interesting analysis. However, if we are looking at the strongest dynasty in 2018, which seems to the be implicit basis of the analysis, then the only relevant metric is the “end year” in 2018. Based on the chart, 254 and 1114 are tied with 8 apiece starting from 2002, and from 2005, its 7 for 254 and 6 for 1114. If we add 2nd rankings, then 1114 has 3 more and 254 has 4 more overall and 2 more since 2005. From 2008 to 2017, 1114 clearly was the dominant dynasty on the basis of this method, but that flipped pretty dramatically in 2018.
The “main triangle-shaped chart” shows which team, for a given start year and end year, had the highest average Elo within that range of years.
So for example, according to Caleb’s method, 254 had the highest Elo when looking only at years 2002-2012, but 1114 had the highest Elo when looking only at years 2003-2012.
I’d say 254 and 1114 are about neck and neck right now if you ask which has the stronger active dynasty. That is, year ranges that end in 2018. 254 takes an edge in the recent year ranges (2012+) years as well as the really old (2002-) year ranges (before 1114 existed), with 1114 still holding a solid claim to the intervening years.
If I understand your methodology correctly, you are averaging stddev-adusted ELOs over consecutive seasons. This is pretty sensitive to extreme outliers. For example, if you pretend a given team had built Simbot SS in 2008, but a median robot every other season, I think that team would wind up being your top choice for a number of cells simply by virtue of how dominant 1114 was in 2008.
What if it I slice it by Championships won? 1114 only has one (2008). 111, 67, and 148 have three. 254 and 71 have four. In virtually every other sports, “dynasties” are measured in Championships, not continued excellence. That’s what separates the modern Patriots from the Buffalo Bills from the early 90s or Philadelphia Eagles of the early 2000s. If you factor in Championships won, 1114 looks a lot more like the Cleveland Cavaliers than the Golden State Warriors (consistently great, but only a single championship).
Moreover, even if we lump in into 1114’s dominant timespan, that’s the same time period in which 67 won all three of their Championships (2005, 2009, 2010). They even beat 1114 in the 2010 finals (although 1114 has twice topped 67 on Einstein - 2008 and 2014). If anything, I view 67 as the dominant dynasty of the 2005-2010 timespan.
This piece is interesting, however I have one major qualm in regards to it. There is a severe bias towards the middle years (2008-2012 in particular) due to the usage of time-spans, out of 153 time-spans compared 2010 was part of the calculation in 81 (or 52.9% of them) while 2018 is only involved in 17 (11.1%) of them. The year 2010 is involved in 4.764 times as many calculations as 2018 is, and since each comparison is strictly a “who’s best” comparison, 4.764 times as valuable to be better at. Based on Caleb’s elo 1114 was better every year from 2006-2013 while 254 was better from 2003-2005 and 2014-2018 resulting in 8 “superior” years each, in those years that 1114 was better they contributed to an average of 75.5 time-spans (or 49.3%) while 254’s years contributed to only 43.5 time-spans on average (or 28.4%) resulting in 1114’s victories carrying 1.735 times as much weight as those of team 254.
For each season I multiplied each team’s elo by the percentage of time-spans each year was involved in, resulting in a weighted elo of 750.35 for 1114 and 734.85 for 254. Using this intentionally biased metric team 1114 has about a 2.1% lead over 254, which seems small, but is slightly more than the difference between teams 148 (elo 1929) and 4539 (elo 1892) from last season. When applying a simple average of elos over the last 16 seasons team 1114 holds a much slighter lead of about .5% (1899-1890) a difference akin to that of 118 (elo 1938) and 148. These less 2008-2012 centric data are close enough that I think declaring one team significantly more dynastic than the other is premature.
If you were to ask any given person who the best all time team in the NFL/NBA/MLB was, they would probably answer based on who has won the most championships. But the 538 article linked by the OP defines a dynasty as continued excellence, and since Caleb’s data is loosely based off that article, I think it’s pretty clear that these results are based off of continued excellence. And I think most sports analysts would agree with that being the best measure of a dynasty.
In virtually every other sport, there aren’t 3800 teams. Am I saying that being the top 1 of 32 teams is equivalent to being the top 120 of 3800 teams? Not necessarily. But it’s something else to think about.
(Actually, the fact that there are so many teams in FRC makes repeated championships even more impressive.)
Having watched the Detroit Red Wings flail around for the last 5 years of their 25 year playoff dynasty, I can attest that they did not continuously reign but are still considered a dynasty.
I think you can’t evaluate FRC dynasties by championships won during a specific amount of time because FRC doesn’t have a salary cap or similar mechanism that creates a finite window of time for success.
As much as it pains me to say this, the Chicago Blackhawks winning 3 cups in 5 years is impressive because they had a limited window to retain the talent required for that success.
In FRC no one is forcing great mentors to move on after they build an Einstein contending team for 5 years straight, so a team that is feared every year for 15 years with a championship here and there is much harder to achieve than a splash team that wins 3 in 5 years but is mediocre over the rest of it’s history.
I think the first scenario speaks to a culture of success and sustainability within a program that transcends any individual, whereas the second scenario typically comes from a couple of wonder mentors bringing success to a team before moving on and taking the success with them.
I always struggle to find the right balance of how many of these posts to directly address. Too many and I come off as defensive, and too few and I come off as unresponsive to criticism. I try very hard not to be either of those things, so if you want a direct response from me feel free to dm me or post twice.
I feel the mean is more applicable than the median when evaluating dynasties. Even though mean is more susceptible to outliers, when we are looking at the best teams we are essentially by definition looking at outlier performances, so it seems silly to me for this to be a large concern. My Elo model wasn’t even designed with the primary purpose of sorting the best teams, but rather to make the best match predictions (which essentially means sorting the mid-range teams), however since the outlier teams are the most fun to look at, I do my best to make it work.
Looking at the mean over a multi-year range is equivalent to asking “which team has on average had the best performance in this time period”, while looking at the median is equivalent to asking “which team’s average performance in this time period was the best”. I personally feel the former is better suited for looking at dynasties, even accounting for outliers. For the curious, here’s what the world chart looks like if we use median instead of mean:
Really not much has changed. The first big change compared to the average chart is that I have 1114 in the 2002-2010 through the 2002-2018 ranges. This is mostly just due to how I defined a team’s score in a year they didn’t exist (essentially considering it to be -infinity). It’s really astounding that even with a year missing, 1114’s median score in that range still beats anyone else. The other notable change you see is more 2056 in the longer year ranges starting in 2009-2012. Still looks pretty clear to me that 1114 is on top though.
I don’t actually see this as an issue. When talking about dynasties, we are referring to continuous performance. A team that does well in 2005-2007 and 2012-2014, but poorly in the intervening years, is less of a dynasty in my opinion than a team that does well in 2008-2013 and poorly in the years before and after. Even if these two teams have the same average performance in the “good” and “bad” years, I feel it is very relevant that the good years be grouped closely together to call something a dynasty.
Because I started my analysis at 2002, and current year is 2018, 2010 does appear to be the most crucial year since it is in the most year ranges on that time period. That’s not locked in though, in 2 more years, 2011 will appear to be the most crucial, and so on. If you set an upper bound on how long an FRC “dynasty” can actually last, these “crucial years” become less relevant. For example, if you say a “dynasty” can last no longer than 15 years, then it’s nonsensical to talk about the year ranges 2002-2017, 2002-2018, and 2003-2018, so 2010 becomes less pivotal to the discussion because it was pulled out of three ranges. I’m not sure what to say the upper bound is for FRC, but for the NFL at least, I think you’d be hard-pressed to say it’s over 20 years, since if you look at the Pats in the 538 article, their so-called 25-year dynasty includes a laughable 2-14 season.
The team_lookup tab in the raw data sheet is interesting to play with. Did you know that by your metric of number of std-devs away from the mean, 1114 has had 3 better seasons than their 2008 season?! (2007, 2010, 2015).
Another interesting tidbit:
In 2008, 1114 scored 7, and the next best team scored 5. In 2007, 1114 had 7 and the next best team had 6. 2 and 1 std dev’s better than the next team.
I wonder what this graph would look like if you squashed the space of the scoring metric. Maybe: the ranking of a team in a year by your previous metric, giving some linearly decreasing number of point as you go from 0-100 or something. The bottom of the chart would look exactly the same (duh) but I bet you might get a bit more 2056 and 254 peppered in that 1114 run.
I wondered what this chart would look like if you just looked at winning percentages. Using @BrennanB’s win rate spreadsheet, I took all of the win rates and averaged them across the year ranges. As I was making this chart, I realized that since I was taking an average win rate, this would penalize teams for not competing as long as other teams (win rates of 0). But maybe that’s OK, because that highlights the sustained excellence of teams. 254 showed up more, because they’ve been playing longer and have great win percentages. The gap between 1114 and 254 will close as they both compete for more years. The single year data ranges can pretty much be ignored in this chart, and some of the early data is a bit garbled, but i think it still shows some interesting data.
