Why do teams die: team attrition in 17 graphs

Inspired by some recent threads, I decided to finally finish parsing data I’ve had for a while on what can predict whether team dies. Since the last few years have been weird, I looked at teams the 2017->18, 2018->19, and 2019->20 transitions.

Summary of what I found:

  • Best predictors of a team surviving are strong robot performance (>1 stdev above median, or an elo of 1560-1580+) and a large number of sponsors (>10)
  • Best predictors of a team dying are the team being an international (non US/Canada) team (especially from China, ~5x worse attrition rate than the US), being a rookie team (~2x as likely to die in every country), or only having a single sponsor (almost always a school)
  • Around 10% of rookie teams die in the US after their first year, compared to 5% of veterans
  • 2017 rookies, for some reason, seemed to have an especially rough time
  • For rookies, I am fairly confident that multiple sponsors and a solid robot performance is indicative of mentors/students already familiar with FRC, which is real reason those teams are way more likely to survive than the average rookie

Anyway, here’s all the data, in way too many histograms:

Performance:

Avg season elo taken from Caleb’s database

2017elo (3) 2018elo (3) 2019elo (2)

year median season elo of teams that died median season elo of teams that continued
2017 1448 1494
2018 1438 1492
2019 1452 1504

Number of sponsors

Number of unique sponsors listed in the team’s official name

2017sponsors 2018sponsors 2019sponsors (1)

year median # sponsors of teams that died median # sponsors of teams that continued % of teams that died that had 1 sponsor
2017 3 6 30.6%
2018 3 6 30.4%
2019 3 5 28.9%

Age of team

Rookie year listed in TBA’s API

2017rookieyear 2018rookieyear 2019rookieyear (1)

year median rookie year of teams that died median rookie year of teams that continued % of teams that died who were rookies
2017 2015 2012 27.3%
2018 2016 2012 26.4%
2019 2017 2013 19.2%

Rookie teams

It seemed that so far, number of sponsors and being a younger team predicted a team’s death the most. Wondering if there was a correlation between the two, I dug more and ended up finding some pretty interesting trends.

2017rookieelo (1) 2018rookieelo 2019rookieelo

year median season elo of rookies that died median season elo of rookies that continued
2017 1443 1449
2018 1432 1440
2019 1442 1456

2017rookiesponsors 2018rookiesponsors 2019rookiesponsors

year median # sponsors of rookies that died median # sponsors of rookies that continued % of rookies that died that had 1 sponsor
2017 1 4 53.3%
2018 2 3 47.1%
2019 1 3 50.9%

Looking at the sponsor lists of these teams with only 1 sponsor (in almost every case, the school) led to what I believe is by far the best predictor of a team’s death: country. Stats for countries with at least 10 rookie teams each year:

country

This trend holds, even when looking purely at veterans:

veteran (1)

Notes:
[1] Teams that did not play in an official event are excluded from the elo calculations
[2] Rookie year plots exclude teams that played in an offseason before their first official rookie season
[3] Number of sponsors were counted from the number of slashes and ampersands in the team “name” field in the TBA API, and may not be fully accurate

71 Likes

Thanks for compiling this data, Rachel!

This is all super interesting, but the most important takeaway for me was the following:

All these teams most commonly die 2 years after their rookie season, which coincidentally is when they lose a lot of grant options. Clearly only supporting teams up to their second year isn’t a successful strategy because a lot of them die off after that. This isn’t anything new to most people, but it’s good having numbers to back that up.

48 Likes

Certainly interesting data… but I suspect the list of reasons are more complex. For example, I’d want to know things like:

  • student membership per team per year
  • mentor membership per team per year
  • school size
  • urban/suburban/rural
  • school-based or community-based
  • proximity to local events
  • health of the FLL and FTC in the team’s area

The data we have access to is only a small piece of the puzzle - it presents correlation, not causation. While that can certainly be used to identify some trends, it can’t be used to identify all of them! It’s really important when looking at what we do have to remember this, so we don’t hyper-focus on one area of attrition while allowing another area to go unnoticed.

For example, rookie teams are more likely to have lower numbers of students and mentors, based on my experience. In a saturated region, they are more likely to come from smaller rural schools or be community based. They are more likely to have longer distances to travel to their first event (excepting teams that form immediately following the introduction of an event in a new location!). If we were able to isolate these factors (we don’t really have the data to do so), we might see some of them have even higher correlation than what’s been found here.

7 Likes

I would like to see this data based on location.

5 Likes

Proportional elo graphs (thanks @asid61 for the suggestion), where you can more clearly see the trend between team performance and teams dying. As I previously said, I believe this is more correlation than causation (teams with a strong structure likely to do well, not a bad season immediately killing a team), but it is still interesting to see:

Veterans:

image image image

Rookies:
image image image

2 Likes

Nice work!

Number of team in China was increasing really fast but,
Unfortunately many teams in China die because of 2019 chinese regionals cancellation.

Regards,
Szymon

4 Likes

Agreed. My first thought here was how grants or external non-sponsor money (I’m thinking of all of the Michigan-specific grants available) impacted rookie team survival rate.

I have to imagine this would be the overall observation - it’s easier for first/second year teams to find grants, but gets harder as you age out of new team grant territory.

2 Likes

I’m curious how you came to that conclusion - what do you see in this that makes country a predictor of rookies failing. It would have been nice to see this as percentages, but to me it seems that, with the exception of Mexico (seriously, congrats Mexico!!!) the attrition rates are relativelyyyy similar between countries overall. I feel like counting China in this is unfair though because they’ve had a series of seriously unfortunate calamities affecting them, unlike any other region in the world.

8 Likes

I’ve been working on a project to see how the income and other factors associated with a team’s zip code affects factors such as travel cost and overall district points.

It’s not as useable or pretty as I would like, but the beginnings are here on my git hub, and some of the charts have been put in this very rough google doc. After I finish the semester, I plan on sharing this data in a much more valuable format.

5 Likes

On a separate note of China, I’ve seen many Chinese teams listed on TBA, and many of them don’t have events listed. What’s with this? Pre-Covid, of course. Here are some teams: 4411 4813 4831 5303 5305 5309
Whatever happened with them, frc.events doesn’t list them as a team.

Could be Off-Seasons. China’s Off-Seasons aren’t much different from a regional, you just don’t go to championships out of them.

4 Likes

@AlexBurchard can obviously speak to this better than I can, but I believe this is similar to many of the Turkish teams before they established their own regional. Teams would register to get the kit of parts, never register for an event, and then Turkey had a big off season even in Decemeber where the teams would compete (including for things like chairman’s and other awards that wouldn’t be normal for an off season event). There was no advancement to championship, but it built a core of experience and teams that was then able to transition to a “traditional” regional model.

2 Likes

Yeah, it’s just confusing since I’m used to events being logged with the team. As an example, 6300 is a team that attended an offseason prior to their official rookie year.

When I started looking into this awhile ago, I was curious about if there could be anything said about team life expectancy and ~4 years (ie maybe you’ve got a particular set of parents + kids that are key to pushing for team survival). My hypothesis was that a majority of teams that fold do within the first 4 or 5 years and would come pretty close to leveling off after that point. My hypothesis wasn’t quite right, but the data is still interesting. This is the data I’ve collected and updated in 2020:

https://drive.google.com/file/d/1j8l7L_eicYVaUbVbG2kyitDuVQF8L8v6/view?usp=sharing

image
This chart is the TL;DR if you don’t want to go to the link. A logarithmic trendline seems to fit pretty well for whatever that’s worth. This graph takes into account that if you want to look for teams that survived for say at least 5 years, you should only look at teams that started in 2015 and before.

Code should be here: TeamAgeData · GitHub

5 Likes

Turkish teams didn’t have to register with FIRST to get a KOP. Fikret Yüksel Foundation gave all new teams a KOP for the off-season for $500 (and we gave them numbers well above FIRST ones of the year the off-season was taking place until they got real FIRST numbers - ex: 6064 was 7011 in the first Turkish Robotics Off-Season, Flare (now 8054) was 8001 from the 2016 off-season until the 2019 off-season, and became 8054 in 2020. We have maybe had a couple teams register for official events, and not take possession of their KOP because of being unable to pay customs fees, and only one team took a KOP at kickoff successfully and failed to compete at a regional, but they came back the next year as a real team and competed, and won. That was due to issues with their school. It’s a cool team in the center of the world actually :stuck_out_tongue: But anyways.

You cannot actually get an official KOP unless you register for an event, and pay your registration, so registering just to get a KOP can’t really be a thing.

2 Likes

Registering for an event != attending an event. It’s possible to register for an event, get KOP, and no-show for various reasons. USUALLY it’s not intentional.

That is possible yes, but very very rare - though what I was responding to was “register, not register for an event, get a KOP, and never compete”, which AFAIK is not possible.

1 Like

That’s why I deferred to you, thanks for the clarification. I wasn’t sure of the specifics.

Any way around it the growth in Turkey has been insane to watch as an outsider.

2 Likes

Teams usually die because of lack of money or lack of mentors.

One common thing I’ve seen is teams that are started by and really pushed by a parent and their kid, and then die shortly after that kid graduated. Nothing against those parents, FRC is taxing, especially when you’re the main driving force, but that’s another one of those barriers teams need to cross before they become self-sustaining.

Can the team fund itself once the rookie and second-year money fizzles out?
Does the team have continued mentorship from multiple committed mentors?

These things are often correlated with # of sponsors or team success, but aren’t the main reason - e.g. helping a team be more competitive isn’t necessarily going to help them sustain their team in FRC.

Three thoughts
(1)

What is the attrition rate for rookie teams that did participate in an off-season before their rookie campaign?

(2) Is there data that looks at teams that died after having played in an offseason that summer/fall?

(3) Perhaps this is better for a different thread, but I’m curious about the scorched earth left behind. I know of several school systems that got burned by the JC Penney wave, and vow never again. Is there data for teams that have rekindled after having taken a season (or more) off, and what were some of the factors that led to the regrowth? I would assume a cycle of administrators and teachers with previous bad experience would help; I also wonder about the restorative power of younger siblings who saw their family members do a cool thing for a year or two, and they try to get the program up and running again. I have a hypothesis that there is a magic amount of time in which (under the right circumstances) teams can rise up again, in a more sustainable way.