How biased are CD polls
 

Partially inspired by this post, I want to know how reflective CD polls are of the FRC community. I will use competitive advancement in the 2015 season as my metric because these results can easily compared against actual team advancement.

If you normally participate in polls, please participate in this one. If you do not normally participate in polls, please do not participate in this one.

Re: How biased are CD polls
 

Using the 2834 scouting database, here is what I found for teams' first event in 2015:

N teams = 2872
N not in eliminations = 1564
N quarterfinalists = 687
N semifinalists = 313
N finalists = 155
N winners = 153

P not in eliminations = 54.5%
P quarterfinalists = 23.9%
P semifinalists = 10.9%
P finalists = 5.4%
P winners = 5.3%

Given, some of those teams do not exist from last year, and this doesn't account for rookie teams or non-FRC members of CD.

Re: How biased are CD polls
 

I can say that my team, which has been around since 2009, joined chief Delphi this year. There's just me and a general team account.

(We have used it as a resource but have never posted anything.)

Re: How biased are CD polls
 

I answered "other" as my team was part of the week 1 Phyxed-Red-Cards alliance. We were technically eliminated in the semifinals, but actually shouldn't have been due to an errant red card...

Re: How biased are CD polls
 

Wow, interesting. The results so far reflect what I've thought for a while: the average Chief Delphi poll-taker is from an above-average team.

This might put the low bar polls into perspective. Most of the teams voting are above-average. That, to some extent, helps justify the extra design challenge of the low bar.

Re: How biased are CD polls
 

We have a few students with CD accounts who check posts and "lurk". The only people to actually post are myself and our engineering mentor. Generally we see CD as a place where the hyper-involved share most of the material and the the soon-to-be-hyper-involved cut their teeth.

Re: How biased are CD polls
 

This is an interesting poll, and will probably show that a disproportionate number of teams here play in elims. But it's also just a single data point, looking at a 1-year view. If you take my team, for example... We did horrible last year. But prior to that, we pretty much always made it into elims. You'll have outliers like that - teams that normally do well but had an off year last year, or teams that made it to elims for the first time in team history last year. Something to keep in mind when looking at the results.

Re: How biased are CD polls
 

+1

4607 fits this description to a T. Recycle Rush wasn't kind to us.

Re: How biased are CD polls
 

I would say the data here may not apply to all teams everywhere but when you look at just Championship only over the course of several years then the data stacks up better to what I would expect from CD polls. For instance, North Carolina is now in a District event model instead of a Regional event. This throws the metrics way off for areas like this because we have multiple district events and a state championship before the big show and therefore more winners.

Re: How biased are CD polls
 

Thanks for creating this! The scouting data and combined with these poll results should be really interesting. So far the data here shows that I was wrong. It will be interesting to see how this poll develops as more people get home from school or work and contribute.

Assuming that the percentages stay the same, roughly 80% of Chief Delphi making eliminations (35% the finals?!) versus 50% of the population of FIRST in general making eliminations is pretty impressive.

Re: How biased are CD polls
 

If we assume that a fixed percentage of all FRC students and mentors are active on CD, having a larger team will disproportionately give them more votes.

On top of that, successful teams often have a greater percentage of CD members that are active then teams that are more the norm.

Re: How biased are CD polls
 

I think that's justified. This poll is measuring how well all of the "voices" on CD do. So if there is a thread about the low bar with 30 people participating, but they are only members of 18 different teams, each of the participants get representation in polls such as this one.

Re: How biased are CD polls
 

A couple more things that this poll does not account for:

Because this is not a public poll, the honesty of the individual voters cannot be verified.

The psychology of the Chief Delphi uses likely makes people who did poorly in their first event last year less likely to vote on this pole, and those who plan on doing a lot of things more likely to vote on the other polls about robot capabilities.

Re: How biased are CD polls
 

I've always taken it for granted that Chief Delphi posters are from above-average teams. That's a good thing when you need technical help, but it also means you have to be careful when you see people talking about strategy. Teams whose goal is to play on Einstein think about strategy differently from teams whose goal is to be selected for an alliance at a district event.

Re: How biased are CD polls
 

That's the entire point of this poll. To show that any poll on CD is inherently biased because the best and large teams are disproportionately represented.

Just about every poll has at least 3 votes from 254. We'd need 9,000 responses to get that to even out the voices of all FRC teams. Based on Caleb's original post we should expect to see 54.5% of teams not making elims last year. As I'm writing this post there are over 200 responses and only 20.0% did not make playoffs. It's not a surprise that CD is biased, but it's cool to have this poll to refer back to years down the road when everyone get's bent out of shape from the responses of a different poll.

Re: How biased are CD polls
 

Also, keep in mind that this is only recording a team's first event. We got knocked out in quarters at our first event because of some really bad luck, then made it to finals at our second event, won chairman's and went to Einstein. The first event alone is hardly a metric for a team's overall performance in a given season.

Re: How biased are CD polls
 

Would you count a backup team going to a final (without being in any other match in that events playoffs) count as being eliminated in the final? If not then it would be just quarterfinals for us.

Re: How biased are CD polls
 

The CD community is just a small very vocal sampling of the overall FIRST community. Many are either too intimidated or think CD is just a colossal waste of time to participate. CD should not be a representative of the pulse of the FIRST community in any way, shape or form.

Re: How biased are CD polls
 

I believe Chief Delphi-posting teams are

1. Better than the average team
2. Not as good as they think they are*

*obvious exemptions apply

Re: How biased are CD polls
 

I personally would count this. If you look at my numbers that I ran in my second post, you will see that there were two more finalists last year than winners. Your team is probably one of those, so I think you have every right to say you were a finalist.

Re: How biased are CD polls
 

I believe it is fair to ask for results in the first tournament since there are teams that only participate in one event a year. As others have mentioned, CD is just a subset of the overall FIRST community and the group tends to come from teams that do better than average. Probably better than average mentor and sponsor support as well.

Re: How biased are CD polls
 

Does anyone have a good estimate for the number of 2016 rookies?

Re: How biased are CD polls
 

After 318 votes, I think the results are pretty accurate for what the CD user base is.

Average CD user probably is in the range of the 40th-80th percentile of performance.

Lots of hive mind here, and frequent repeated mistakes in design and strategy, as evidenced by the Karthik scared discussion. The highest performers buck the trend continuously.

Re: How biased are CD polls
 

Devin, ditto! I was just going to say the same thing. We have at least a half dozen from our team posting on CD...

Re: How biased are CD polls
 

That's exactly what we're trying to show with this poll. That poll's aren't a good representation of average teams because the biggest and elite teams have many members filling in the same poll skewing the results.

Re: How biased are CD polls
 

When has any online poll been a good representation of anything?