How biased are CD polls

[Caleb Sykes]

Partially inspired by this post, I want to know how reflective CD polls are of the FRC community. I will use competitive advancement in the 2015 season as my metric because these results can easily compared against actual team advancement.

If you normally participate in polls, please participate in this one. If you do not normally participate in polls, please do not participate in this one.

[Caleb Sykes]

Using the 2834 scouting database, here is what I found for teams' first event in 2015:

N teams = 2872
N not in eliminations = 1564
N quarterfinalists = 687
N semifinalists = 313
N finalists = 155
N winners = 153

P not in eliminations = 54.5%
P quarterfinalists = 23.9%
P semifinalists = 10.9%
P finalists = 5.4%
P winners = 5.3%

Given, some of those teams do not exist from last year, and this doesn't account for rookie teams or non-FRC members of CD.

[maxnz]

I can say that my team, which has been around since 2009, joined chief Delphi this year. There's just me and a general team account.

(We have used it as a resource but have never posted anything.)

[MechEng83]

I answered "other" as my team was part of the week 1 Phyxed-Red-Cards alliance. We were technically eliminated in the semifinals, but actually shouldn't have been due to an errant red card...

[Dinokaiz2]

Wow, interesting. The results so far reflect what I've thought for a while: the average Chief Delphi poll-taker is from an above-average team.

This might put the low bar polls into perspective. Most of the teams voting are above-average. That, to some extent, helps justify the extra design challenge of the low bar.

[SenorZ]

We have a few students with CD accounts who check posts and "lurk". The only people to actually post are myself and our engineering mentor. Generally we see CD as a place where the hyper-involved share most of the material and the the soon-to-be-hyper-involved cut their teeth.

[Jon Stratis]

This is an interesting poll, and will probably show that a disproportionate number of teams here play in elims. But it's also just a single data point, looking at a 1-year view. If you take my team, for example... We did horrible last year. But prior to that, we pretty much always made it into elims. You'll have outliers like that - teams that normally do well but had an off year last year, or teams that made it to elims for the first time in team history last year. Something to keep in mind when looking at the results.

[Ginger Power]

Originally Posted by Jon Stratis

This is an interesting poll, and will probably show that a disproportionate number of teams here play in elims. But it's also just a single data point, looking at a 1-year view. If you take my team, for example... We did horrible last year. But prior to that, we pretty much always made it into elims. You'll have outliers like that - teams that normally do well but had an off year last year, or teams that made it to elims for the first time in team history last year. Something to keep in mind when looking at the results.

+1

4607 fits this description to a T. Recycle Rush wasn't kind to us.

[OccamzRazor]

Originally Posted by Jon Stratis

This is an interesting poll, and will probably show that a disproportionate number of teams here play in elims. But it's also just a single data point, looking at a 1-year view. If you take my team, for example... We did horrible last year. But prior to that, we pretty much always made it into elims. You'll have outliers like that - teams that normally do well but had an off year last year, or teams that made it to elims for the first time in team history last year. Something to keep in mind when looking at the results.

I would say the data here may not apply to all teams everywhere but when you look at just Championship only over the course of several years then the data stacks up better to what I would expect from CD polls. For instance, North Carolina is now in a District event model instead of a Regional event. This throws the metrics way off for areas like this because we have multiple district events and a state championship before the big show and therefore more winners.

[Keefe2471]

Thanks for creating this! The scouting data and combined with these poll results should be really interesting. So far the data here shows that I was wrong. It will be interesting to see how this poll develops as more people get home from school or work and contribute.

Assuming that the percentages stay the same, roughly 80% of Chief Delphi making eliminations (35% the finals?!) versus 50% of the population of FIRST in general making eliminations is pretty impressive.

[RoboChair]

If we assume that a fixed percentage of all FRC students and mentors are active on CD, having a larger team will disproportionately give them more votes.

On top of that, successful teams often have a greater percentage of CD members that are active then teams that are more the norm.

[Monochron]

Originally Posted by RoboChair

If we assume that a fixed percentage of all FRC students and mentors are active on CD, having a larger team will disproportionately give them more votes.

I think that's justified. This poll is measuring how well all of the "voices" on CD do. So if there is a thread about the low bar with 30 people participating, but they are only members of 18 different teams, each of the participants get representation in polls such as this one.

[Bryce2471]

A couple more things that this poll does not account for:

Because this is not a public poll, the honesty of the individual voters cannot be verified.

The psychology of the Chief Delphi uses likely makes people who did poorly in their first event last year less likely to vote on this pole, and those who plan on doing a lot of things more likely to vote on the other polls about robot capabilities.

[alopex_rex]

I've always taken it for granted that Chief Delphi posters are from above-average teams. That's a good thing when you need technical help, but it also means you have to be careful when you see people talking about strategy. Teams whose goal is to play on Einstein think about strategy differently from teams whose goal is to be selected for an alliance at a district event.

[pntbll1313]

Originally Posted by RoboChair

If we assume that a fixed percentage of all FRC students and mentors are active on CD, having a larger team will disproportionately give them more votes.

On top of that, successful teams often have a greater percentage of CD members that are active then teams that are more the norm.

That's the entire point of this poll. To show that any poll on CD is inherently biased because the best and large teams are disproportionately represented.

Just about every poll has at least 3 votes from 254. We'd need 9,000 responses to get that to even out the voices of all FRC teams. Based on Caleb's original post we should expect to see 54.5% of teams not making elims last year. As I'm writing this post there are over 200 responses and only 20.0% did not make playoffs. It's not a surprise that CD is biased, but it's cool to have this poll to refer back to years down the road when everyone get's bent out of shape from the responses of a different poll.