FRC Blog - How We’re Doing and FIRST Babies

[Nathan Streeter]

Originally Posted by dodar

Use some common sense man. You wanna know the effect of rules on a game, ask the people who deal with them directly on a constant basis.

And those front office people are only on there because of NFL politics.

I think common sense was used.

There is much more to the success of an FRC game than whether or not the 4 individuals per team around the field thought it was the best game. How about things like student and team retention rate? How easy it is for "outsiders" to watch? How interesting is it for experienced FIRSTers to watch? Is the field affordable, rugged, and able to be reset quickly? The subjective, "Are students inspired?" aspect? Now, I'm not saying Aerial Assist is the best game by all these measures, but it certainly isn't the horrendous game that some people are trying to pretend that it is.

I am honestly tired of hearing people complain about this game... so I guess I'll join the party of complaints by complaining about the people complaining. This game isn't bad.*

- This game provides a way for simple, effective robots to do surprisingly well. I've seen plenty of robots here in New England, particularly ones made by rookie teams, that were designed to do a limited skill set very well... and they've been quite successful. No, not more successful than the more complicated (and effective) robots, but still enough to do very well in their role.

- You may complain about qualification matches being too dependent on your partners' capabilities, but I've seen the WLT ranking system do a great job at putting the best robots in/near the top. This may be partially due to the district system (high match to # of teams ratio), but it's done a very good job. Good teams win more qual matches, even with the occasional tech foul, delayed pedestal, or inconsistent partner. If qual matches are such crap-shoot how are the best teams at many events able to go undefeated?

- Game pieces are robust and (sufficiently) available. Looking back at Rebound Rumble, Logomotion, Rack 'n Roll, or Aim High with rose-tinted glasses? How about the fact that these game pieces actually survive events and don't vary as soon as "fresh" game pieces appear? Yes they can pop, but I think it's probably happened in our team's 38 matches only once.

- Match turnaround is straight-forward and quick (after the first dozen matches). 'nuff said.

- Matches are strategic... match strategy varies and there are several viable strategies for winning depending on your alliance's composition. You don't need to truss and catch and shoot high and get triple assists. You can choose one or two elements and play to your alliance's strength (although I'm guessing this will decline as you get to higher levels of play).

- Auto has a significant impact on the match outcome, but is by no means a lock or insurmountable obstacle.

So there you have it... a few things that are positive about this game. Try to mix things up and think of some more positive things rather than just seething over the fact that you lost matches over weird calls, missed assists, or feel let-down about how your season ended. No, this game isn't perfect... but neither were any of the prior games and don't expect that to change.

Originally Posted by Madison

However, the field does not work and the referees are not able to credit teams appropriately for accomplishing rudimentary game tasks. Can anyone reasonably refute either of these claims?

I disagree with both of those claims. At the 4 events I have attended the field and the referees have generally done a good job; malfunctions or botched calls have been uncommon, not the norm. Yes, week 1 events were rough around the edges, but that's hardly enough to ruin a game.

Originally Posted by magnets

In fact, I am wondering if there is a single drive coach (with a few years of FRC experience) of a competitive, elimination round, non rookie team that is happy with the game.

I haven't drive coached this year (despite being involved with strategy, scouting, and pit work), but I'm happy with the game, and so are other drive coaches I know.

*So you insist, "Well, that's subjective!" OK, well I can insist it's a good game, you can insist it's a bad game... end of story.

For the record, I too find the subjective nature of many rules frustrating. I too think the refs are over-burdened. I too think this game has flaws... that doesn't mean that the sky is falling though.

[Travis Hoffman]

Here's the Week 5 survey link, my people:

https://www.surveymonkey.com/s/FRC2014TeamSurvey-week5

Wouldn't want anyone who competed last week to feel left out!

[DampRobot]

Originally Posted by Travis Hoffman

Here's the Week 5 survey link, my people:

https://www.surveymonkey.com/s/FRC2014TeamSurvey-week5

Wouldn't want anyone who competed last week to feel left out!

Uhh... are you sure they want you sharing this? I was under the impression that they limited the survey to 2/team to try to give their sample a more or less even distribution across teams.

[Travis Hoffman]

Originally Posted by DampRobot

Uhh... are you sure they want you sharing this? I was under the impression that they limited the survey to 2/team to try to give their sample a more or less even distribution across teams.

"Survey for mentors & student team members from FIRST Headquarters is open only until midnight Wednesday April 2"

"please forward link to your team and encourage them to complete it - thank you!!"

We're all on the same team, are we not? Kum-ba-yah.

[Swan217]

For statistical purposes, "What do you think of the game?" is not a useful question. Respondents could be referring to the quality of the rules, but more likely to the overall idea of the game "giant balls being thrown through holes while other robots assist."

Additionally, the respondents are self-selecting, which additionally calls the validity (statistically speaking) of the survey into question. In order to get the most accurate representation, a random sample needs to be taken.

In addition, there aren't any crosstabs available, but all of this is covered under "The usual caveats... apply" paragraph stating that this is NOT a scientific survey. But you don't "tease" this sort of data unless you're trying to quell some serious dissent in the ranks. This is evident in the "burden placed on our volunteer referees" comment, which has to be the largest understatement of the month.

[Travis Hoffman]

Originally Posted by Swan217

In addition, there aren't any crosstabs available, but all of this is covered under "The usual caveats... apply" paragraph stating that this is NOT a scientific survey. But you don't "tease" this sort of data unless you're trying to quell some serious dissent in the ranks. This is evident in the "burden placed on our volunteer referees" comment, which has to be the largest understatement of the month.

The only way to quell serious dissent in the ranks is to implement meaningful change.

c.f. Einstein 2012

This is worse. Ain't just 12 teams denied the appropriate opportunity to shine to their full potential. And is this not a similar situation, where the governing body, via their inaction and/or incomplete attention to design detail, failed to sufficiently catch and address significant flaws that ended up adversely impacting teams? I see no difference here, except instead of wireless communications hardware/firmware, we're talking referee->field interface hardware/software, game/foul rules, and referee training and preparation, and we're talking a significantly greater number of teams impacted.

FIRST took very proactive and well-received steps to correct the Einstein 2012 situation. I expect NO LESS from them in addressing this current situation.

[Ian Curtis]

Originally Posted by Jon Stratis

So you assume Frank is attempting to mislead us by cherry-picking the statistics he shares? Do you have some evidence for this, or is it based on personal bias and a very vocal few on CD?

I think it is a reasonable assumption that anyone who presents you with data is trying to sell you a particular version of a story. The contrast between Frank's wording and IndySam's explanation of the same chart is a great example of how two people can read very different things from the same data!

Originally Posted by IndySam

What I see in this blog post is a 40% reduction of very good ratings and a 400% increase of very poor ratings.

I don't think this is something to brag about.

Originally Posted by Frank

1.2 % of respondents rated the game ‘Very Poor’, while 34.8% of respondents rated the game ‘Very Good’. As a point of comparison with last year’s game Ultimate Ascent, arguably one of our most popular in recent history, 91.3% rated the game positively over all weeks of the regular competition season, with 48.7% rating it ‘Very Good’, while 1.7% rated the game negatively, with 0.3% rating it ‘Very Poor’. (Yes, even for Ultimate Ascent, we had a handful of survey respondents who strongly disliked the game).

...

We’d love for every game to achieve Ultimate Ascent-like popularity, and we did not reach that level this year.

My reading of the data is that a significant portion of the people who respond to optional surveys in FIRST e-mail blasts think that both Ultimate Ascent and Aerial Assist are good games. I would wager that these people would say the same thing of any FRC game, but I of course have no data to back this up.

Now that Travis has posted the link on CD, I expect the week 5 results will be different and no longer representative of people who respond to optional surveys in FIRST e-mail blasts.

[Travis Hoffman]

Originally Posted by Ian Curtis

Now that Travis has posted the link on CD, I expect the week 5 results will be different and no longer representative of people who respond to optional surveys in FIRST e-mail blasts.

G.I. Joe taught me that knowing is half the battle.

[wilsonmw04]

This tread sounds like some of my students after they get a test back. These students who have not done well and complain that the test was "a bad" test for various reasons. I would then post the range of test scores. They would not be significantly different than the results of the last test. The students would still insist the test is bad or the teacher didn't teach the material properly.

It's amazing how folks can see data right in front of them and still insist their original opinion is that of the majority when it may not be.

So much rage this year. So much misplaced rage.

[martin417]

I would love to see a survey of a group of people who had no stake in the outcome of any one match, but were absolutely very involved with the play and execution of every match: the referees.

Here is an unbiased group that can let their feelings be known without being accused of sour grapes or team bias.

Why not ask them?

[billylo]

Originally Posted by martin417

I would love to see a survey of a group of people who had no stake in the outcome of any one match, but were absolutely very involved with the play and execution of every match: the referees.

Here is an unbiased group that can let their feelings be known without being accused of sour grapes or team bias.

Why not ask them?

^^^ This ^^^^

Many refs are long-time, committed FIRSTers who are generally a rational bunch.

To the refs: your perspectives can shine light on making the next game better.

[Chris Fultz]

Anyone notice the data makes a little bit of a smirk/smile, and it was posted April 1?

Just wondering.

[Nick Lawrence]

Originally Posted by martin417

I would love to see a survey of a group of people who had no stake in the outcome of any one match, but were absolutely very involved with the play and execution of every match: the referees.

Here is an unbiased group that can let their feelings be known without being accused of sour grapes or team bias.

Why not ask them?

I would wager significant money that their responses would be skewed towards the negative, due to the fact that they were overwhelmed all year. I think that the results would be fair and very indicitative of this game's true colours. If the refs aren't happy and confident, the game just doesn't work, since they directly control match flow. I in no way blame the individual referees. I know many who are some of the best in the business, and even they are overwhelmed.

This year was a very good experiment for the FRC GDC and FIRST as a whole. It showed us how some really great ideas don't translate well into portions of FRC games, with some current team cultures. It also showed the true character of some folks this year, through both their frustration and resilience.

-Nick

[Travis Hoffman]

Originally Posted by wilsonmw04

This tread sounds like some of my students after they get a test back. These students who have not done well and complain that the test was "a bad" test for various reasons. I would then post the range of test scores. They would not be significantly different than the results of the last test. The students would still insist the test is bad or the teacher didn't teach the material properly.

It's amazing how folks can see data right in front of them and still insist their original opinion is that of the majority when it may not be.

So much rage this year. So much misplaced rage.

Actually, this season, the teacher decided to randomly yoink the test out of a large percentage of students' hands before the normally alotted amount of time for taking the exam ran out. Each and every bad foul call, pedestal delay, missed assist, unpenalized instance of damage, etc. yoinked additional valuable time away from the students trying to pass the exam.

Some high-performing students may have been better able to still "ace the test" given the artificially compressed environment foisted upon them. Others who very well could have passed the test if given the full allotted time will never know if they actually would have achieved that goal because THE TEACHER NEVER GAVE THEM THE FULL OPPORTUNITY TO SUCCEED THEY WERE ORIGINALLY PROMISED.


Also, the survey email I received from the (very good, nice, and wonderful) Pittsburgh RD also contains the following:

"A note about surveys:
Seriously, we are working hard to NOT over-survey you guys! Both FIRST HQ and I need team data in order to write grant reports and funding proposals. Last year we had only a 27% return on student surveys at the Pittsburgh Regional, and ~1% (yep, one %) return on HQ surveys from our event. The only way we can ensure having regional competitions is to have evaluation processes in place that work. I thank you from the bottom of my heart for your patience and care in responding to our surveys."

I don't know what a good number is for an estimate of the total students/mentors in FIRST, but let's assume (very conservatively) 20 students/mentors per team on average. 20 * 2700 = 54000 potential responses. 3600 responses out of a possible 54000 is a mere 6.6%. And this doesn't include the fact teams are given the chance to respond multiple times if they attend multiple events.

Honestly, I'd like to see more team members SPEAK UP and contribute to these surveys regardless of opinion, such that the statistics drawn from them become more relevant and worthy of reporting.

[martin417]

Originally Posted by wilsonmw04

....So much rage this year. So much misplaced rage.

Shouldn't the fact that there is so much rage this year mean something? In my time participating in FIRST, have never seen so many people that really don't like the game. If everything was truly rainbows and unicorns, there would be a lot fewer dis-satisfied customers.