Time for a Scouting Award--Eagle Scout???

[MechEng83]

This would be a nightmare to judge. Judges are recruited from the professional world. Often, they have never seen an FRC competition or know what FIRST is prior to walking into the first judges' meeting. When they judge things like machine design, they're drawing on their knowledge base and generally have a good grasp on mechanical/electrical/computer systems. Other judges are recruited for their expertise in things like business planning or social awareness.

IMHO, to judge this would require a panel of judges who are intimately familiar with scouting practices, ranking systems, and how scouting fits in to the competition.

Also, I think the creativity award has been given to teams who have a unique scouting system, as this falls under "strategy." There is also the Judges Award if judges feel the team deserves special recognition for their scouting system.

[Citrus Dad]

Originally Posted by MechEng83

This would be a nightmare to judge. Judges are recruited from the professional world. Often, they have never seen an FRC competition or know what FIRST is prior to walking into the first judges' meeting. When they judge things like machine design, they're drawing on their knowledge base and generally have a good grasp on mechanical/electrical/computer systems. Other judges are recruited for their expertise in things like business planning or social awareness.

I'm a professional economist. I use many of the methods used in scouting in my daily work. In fact for many in the financial world, this is their daily task. Operations research (for you engineers) uses these techniques as well. If you've seen or read "Moneyball" you've seen good disciplined scouting in action. Both raising the awareness of scouting and recruiting judges from a wider array of technical fields should broaden the appeal of FIRST.

[MechEng83]

Originally Posted by Citrus Dad

I'm a professional economist. I use many of the methods used in scouting in my daily work. In fact for many in the financial world, this is their daily task. Operations research (for you engineers) uses these techniques as well. If you've seen or read "Moneyball" you've seen good disciplined scouting in action. Both raising the awareness of scouting and recruiting judges from a wider array of technical fields should broaden the appeal of FIRST.

It sounds like you want this award to be a "statistical analysis" award. That is certainly an important part of of scouting and would promote the often neglected "M" in STEM, but so is boots on the ground (or rather, eyeballs on the field), creating a robust data entry system, picking the right selection criteria (how do you judge what is right for this?), and hosts of qualitative information. Does this all become part of the award?

Don't get me wrong, I think scouting is an invaluable part of having a successful run in a tournament, but the effectiveness of a scouting system might not be visible until after alliance selections, at which point awards are (usually) decided and scripts have been written.

[Lil' Lavery]

Which of you have intentionally brought up your scouting system to judges before? Most judges afford you the opportunity to talk about anything interesting they might not have directly asked you about, so it's possible for you to inform them about the dilligence or innovations in your scouting system, and how it impacts your strategy/performance. If it impressed them enough, it could very well had tied into a "Judge's Award" or even a couple technical awards you were already being considered for (namely Quality or Creativity).

[Jon Stratis]

I can see some potential difficulties for a scouting award - chief among them the fact that the scouts are (for the most part) in the stands, not in the pits where the judges are!

But to be productive towards the concept of creating such an award, here are my thoughts:
- Recruit judges from local sport organizations. Those organizations, be it professional or college, have scouting programs of their own. They scout prospective recruits. They scout upcoming opponents. They collect and collate a wealth of information in order to make the decisions they need to make, and someone like that would be an ideal candidate for judging scouting in FRC. Further, it would help us, as a program, reach out into the existing sports world to become better known.
- Look for how a team uses its scouting data. Is it only for alliance selection? Is it used to determine strategy for elims? Is it used the entire competition to determine strategy for every match? Is it used to send students to learn more about efficient, excellent designs?
- Look for how a team uses their scouting data/system for outreach. Is information available to other teams? Do they participate in a multi-team scouting approach? Do they have information on their system (or even a blank copy of the system) available for other teams?
- Look for the quality of product produced. Are they setting a clear separation of qualitative versus quantitative data? What sort of data are they gathering, and how do they sum it up to create something usable?
- Look for efficiency in the scouting process. What has the team done over the past few years to make the process easier and more reliable? How many people does it take to scout a single match, and why does it take that many?

There are many approaches to scouting out there, and many different systems. There are strengths and weaknesses for all of them. Having an award that celebrates scouting could help increase awareness of the need for this data amongst younger teams, and provide an incentive for older teams with established scouting programs to reach out and bring everyone up to speed.

[Citrus Dad]

I absolutely agree with Jon's assessment and points. Getting the judges into stands might connect them with more team members.

[Citrus Dad]

Originally Posted by Lil' Lavery

Which of you have intentionally brought up your scouting system to judges before? Most judges afford you the opportunity to talk about anything interesting they might not have directly asked you about, so it's possible for you to inform them about the dilligence or innovations in your scouting system, and how it impacts your strategy/performance. If it impressed them enough, it could very well had tied into a "Judge's Award" or even a couple technical awards you were already being considered for (namely Quality or Creativity).

Up to this point I believe most pit crews, who talks with the judges, both don't understand scouting system well enough, including its importance, and view the scouts as being less worthy. Let's be honest--most students are in FRC to build and work on robots, not to sit in the stands and scout matches. Certainly most of our scouts would rather be on the pit crew, justifiably so. So I can't expect pit crews to extoll the scouting system over the quality of the robot build. Most often I've seen the Judges Award given to teams that have virtuous robot designs that weren't the top in the individual categories, but maybe the best overall.

[XaulZan11]

Originally Posted by Lil' Lavery

Which of you have intentionally brought up your scouting system to judges before? Most judges afford you the opportunity to talk about anything interesting they might not have directly asked you about, so it's possible for you to inform them about the dilligence or innovations in your scouting system, and how it impacts your strategy/performance. If it impressed them enough, it could very well had tied into a "Judge's Award" or even a couple technical awards you were already being considered for (namely Quality or Creativity).

This. Didn't 3138 win an award at the Championship last year for their IPA scouting system? Our scouting system (and our graciousness in sharing the system and data) was mentioned by the judges when we won the Chairman's award at Midwest this year. I believe our Chairman's presenters brought it up in the presentation or Q&A.

My concern with the "Scouting Award" is it will go to the teams that use fancy tablets or cell-phones and produce cool graphs and fancy statistics opposed to the more user-friendly, functional and practical systems. As Jon mentioned, the judges NEED to look how the team uses the data. There was one team that used video game controllers to scout, which seems to be a really innovative and award-winning system, but they team didn't create a picklist outside of 10-12 teams that stood out.

But, I guess that concern could be applied to the technical awards, too. You rarely see teams win awards for tank drives, even though they are arguably the best solution.

[Citrus Dad]

Originally Posted by MechEng83

It sounds like you want this award to be a "statistical analysis" award. That is certainly an important part of of scouting and would promote the often neglected "M" in STEM, but so is boots on the ground (or rather, eyeballs on the field), creating a robust data entry system, picking the right selection criteria (how do you judge what is right for this?), and hosts of qualitative information. Does this all become part of the award?

Don't get me wrong, I think scouting is an invaluable part of having a successful run in a tournament, but the effectiveness of a scouting system might not be visible until after alliance selections, at which point awards are (usually) decided and scripts have been written.

I absolutely agree that that other elements must be included as well (e.g., team management), as well as innovative data entry systems, communicating to the drive team during competition. There are ways of converting qualitative data into fairly rigorous quantitative data that gets a team down to just a couple of judgments. All of these can go into a scouting award.

To be honest, there are many long-time sports fans who could quickly pick up on which of the scouting systems are most successful. I've only been with a robotics team for 3 years, but I brought my interest in other sports statistics to improve our scouting system. I could walk around in the stands (which I don't think judges do for any other award--I think team spirit is judged from the field) and pretty easily assess which teams have good scouting systems. I think the award should in particular look for innovative and rigorous systems that could be duplicated by other teams. Truly this would be no more difficulty than judging creativity or engineering excellence (and once in a while I wonder why those teams win an award after watching them as a scout... )