Rookie All-Star Data

FIRST General Forum
1

With changes to Regional advancement, there have been some questions surrounding the Rookie All-Star Award and its relation to further success.

Through a series of inefficient API calls and smashing my head against the wall, I have some data to share:

Do Rookie All-Star Award winners remain in FIRST for longer than a typical team?

Short answer: Yes.

Longer answer: There is evidence to suggest that RAS winners are more likely to survive at least three seasons than teams who do not.

Data caveats: More teams win RAS each year as more teams and more events exist. The data is skewed in favor of this hypothesis since more teams have won RAS in recent years. The proportion of teams still in operation today is obviously going to be higher for the RAS teams than all teams, since there are more recent RAS teams than more recent teams total.

Time before disbanding Number of all teams (proportion) Number of RAS-winning teams
<3 years 2328 (31%) 239 (16%)
3-5 years 933 (12%) 127 (8%)
6-8 years 491 (6%) 74 (5%)
9-11 years 238 (3%) 33 (2%)
>=12 years 230 (3%) 25 (2%)
Still in operation 3407 (45%) 1012 (67%)

Causality? I don’t think there is sufficient evidence to say that winning the Rookie All-Star award causes teams to stick around longer in FIRST. A more likely conclusion would be that the RAS award recognizes exceptional rookies, and exceptional rookies are more likely to succeed. A study to determine why teams folded would be fascinating, but there’s no dataset for this, as far as I’m aware.

Do Rookie All-Star Award winners win the Chairman’s Award or FIRST Impact Award more?

Short answer: Yes.

Longer answer: Teams are considerably more likely to win the Chairman’s Award or FIRST Impact Award at some point in their career if they previously won the Rookie All-Star award.

Data caveats: All levels are counted the same (regional, district, DCMP, Championship, etc.) and multiple wins in the season (e.g. at a district event and DCMP) are all counted.

RAS-winning teams All teams
Mean number of CA/FIA award wins per team 0.5 0.3
Percentage of teams with at least one CA or FIA award 17% 9%

Causality? Again, I think not, for the same reasons as above.

The elephant in the room - FIRST Championship

A lot of this discussion was sparked by the question of whether or not a guaranteed ticket to the FIRST Championship for RAS winners as worthwhile. The data cannot prove this one way or another. We can’t compare teams auto-qualifying and not auto-qualifying based on Champs attendance. Perhaps, in five years without auto-attendance, we will have some interesting data to compare, but today is not that day.

Areas for further research

  • Does Championship attendance correlate to team longevity? What about Championship attendance within a team’s first 1/2/3 years?
  • What causes teams to fold (an existential question in FIRST)?
35 Likes
2

As always, very nice data, and a very well written summary. I’d be interested to see similar data, but with the (now retired) highest seeded rookie award.

I feel like this could be your “about me.”

10 Likes
3

Let’s keep the head-wall interactions to a minimum, Jared.

In the last few years there has been a disclaimer that not every event with rookie teams needs to generate a RAS award (not sure if this was always the case). It comes down to the judges at the event making some combination of objective and subjective evaluation of the qualities of the rookie teams. So, with a large enough data set, and consistent enough judging criteria, the RAS winners should show some trend of higher excellence. And it’s good to see that in your stats.

I wonder if the trend holds for “Rookie Inspiration Award” where there’s no payout?

1 Like
4

i feel like the 2010’s JCPenney surge, with lack of financial follow up, and covid drop-offs could potentially play into skewing data?

Many teams shut down from covid, whether RAS winners or not, it threw everything in a loop.

2 Likes
5

Wait what does JCpenny have to do with FIRST?

6

They gave out some substantial grants, then stopped giving out those grants.

Others can probably pull up more relevant discussions on the matter, but here are a few:

3 Likes
7

That’s about the size of it. Maybe 2 years of major-sized grants, then poof no more grants, no more sponsorships, no more teams (at least for a lot of teams–a few did survive for longer).

Even today, there’s a bit of a concern among the veterans that the hardest season a team will face is not their first season. It’s their third season, and then their 5th season. First and second seasons, there’s the rookie grants of various types. EVERYBODY wants to help out the rookies. And a few of them also give a bit to second-year teams, because the transition to “all on your own” is rough. By third year, you’re on your own, but you should have a couple seasons of experience in fundraising the rest of things–if you do, you’re fine; if you don’t you’re in trouble. 5th year is when the last of your rookie season’s freshmen have graduated, so you find out if your lessons are truly being heeded by those that follow…

5 Likes
8

Oh good lordy…

Its worth the search on here, lots of content on that mess.

9

Would it be because of going to champs that causes them to be sustainable or because the team might have a better run/sustainable program that leads them to qualify consistently?

How does average EPA year to year correlate to team sustainability? Do “better” teams actually last longer?

How about spikes in performance from a random motivated student going through the program?

1 Like
10

Hey I might have some ideas about this one

6 Likes
11

Main benefit off winning RAS is the trip to the worlds in the past , This allows that team in its first year to experience hundreds of exemplerary teams at a formative stage of their existence. Its well worth keeping that award going for rookie teams.

2 Likes
12

It would be interesting to run the same numbers for rookie inspiration. That’s presumably the second best rookie team, but without the benefit of a championship trip.

1 Like
13

I agree with Joe. RAS vs RI longevity might be the closest we can get to a control group comparison.

1 Like
14

This is something I’ve been thinking about a lot. Would a guaranteed spot at champs encourage 1st year teams to be more sustainable and impactful? Can a trip to champs be used as an encouragement to start sustainable teams? Does Bean Kamen even care about sustainability, or just inflating the total team count? I know the thought of going to champs in my FIRST year really pushed me to give my all in outreach and judging.

15

Data caveats: The Rookie Inspiration Award was first awarded in 2004, so for the easiest comparison, all three groups only look at teams and events starting in 2024. Also, there are many teams (31% of RI winners) who won both awards. Since Rookie All-Star is considered a more prestigious award than Rookie Inspiration, teams who won both are counted in the RAS group.

Do Rookie Inspiration Award winners remain in FIRST for longer than a typical team?

Short answer: Slightly.

Long answer: There is evidence to suggest that Rookie Inspiration winners are about 20% more likely to continue past their third year. These gains are not insignificant, but are much less than the Rookie All-Star award winning teams who are about 280% more likely to survive past their third year.

Time before disbanding Number of all teams (2004-present) Number of RI-winning teams (excl. teams who also won RAS) Number of RAS-winning teams (2004-present)
<3 years 2045 (31%) 268 (26%) 231 (11%)
3-5 years 793 (12%) 123 (12%) 225 (11%)
6-8 years 413 (6%) 61 (6%) 271 (13%)
9-11 years 190 (3%) 19 (2%) 179 (9%)
>=12 years 144 (2%) 19 (2%) 231 (11%)
Still in operation 3098 (45%) 526 (52%) 968 (46%)

Causality? Doubtful. The correlation here is likely caused by the fact that most teams (about 2/3) make it past the first three years, and the Rookie Inspiration award is awarding a typical team. The teams who do not make it past the first three years are exceptional, and Rookie Inspiration teams are more average.

Do Rookie Inspiration Award winners win the Chairman’s Award or FIRST Impact Award more?

Short answer: No

Long answer: Rookie Inspiration winners (who do not also win the Rookie All-Star Award) are roughly as likely to win the Chairman’s Award or FIRST Impact Award as any other team.

RI-winning teams (excl. teams who also won RAS) All teams (2004-present) RAS-winning teams (2004-present)
Mean number of CA/FIA award wins per team 0.22 0.24 0.44
Percentage of teams with at least one CA or FIA award 7% 8% 15%

Causality? Again, probably not causal, even if correlated. This data suggests that Rookie Inspiration Award winners may resemble typical teams in FIRST. That is to say, they are not exceptional in this way.

6 Likes
16

Thanks for running all these Jared.

I can see merits of keeping the groups seperated, but as you pointed out, the “prestige” associated with RAS vs RInsp and the chance a team could win both kinda messes with the results a bit (or at least makes interpretation less straightforward)

17

This to me is the most telling.

Regardless of accomplishment in the rookie year it seems like all rookies are equally hurt during the time period where they need to survive multiple years without Rookie/2nd year grants, and when your original freshman graduate.

Although the focusing of the data to the last 20 years did something interesting.

When looking at all time data 1012 RAS (67%) are still around but when removing everything prior to 2004 this number drops to 968 (46%). Considering the overall teams is sitting at 3098 (56%) which the RAS are still part of said data set doesn’t this flip your original conclusion?

When focused on the last 20 years doesn’t the data say that RAS are still more likely to survive the first 2 years however they are infact less likely to survive in general than NON-RAS winners long term?

18

I would say with the delta being less than 1% between the RAS winners and all teams that the outcomes are appreciably the same between the two groups in that time period, but yes, certainly not better for RAS teams year 3 and beyond.

1 Like
19

Ah. I think James is commenting on this data point:

which I believe has a typo and should say 46%. Right now, your table looks like it says winning RAS leads to worse outcomes than winning RI, which leads to worse outcomes than winning neither in terms of survival to the present.

1 Like
20

It should in fact be 45%. Doh.

1 Like