It crossed my mind recently that there isn’t an award dedicated to a team’s innovation and creativity with their scouting and data analyzing. While there is an argument to be made that winning a regional/district event is the reward for having good scouting, that isn’t always true. It’s possible for a team to have an incredible scouting system, but get faced with an unlucky match schedule or have a robot that doesn’t perform well enough in qualifications to be an alliance captain, both of which I’ve seen happen to teams in past seasons. Some teams have really unique and well built systems for scouting, which although doesn’t directly impact the function and design of the robot, can still play a huge role in a team’s success, and I think there should be a way of recognizing that.
This leads to my idea for a new award: Innovation in Scouting Award. It recognizes a team for having a well built and innovative system for scouting and data analysis. Its description can be: “Celebrates an innovative an well developed system for collecting and analyzing the performance of other robots.” For the award’s guidelines, they can be:
A team must be able to describe their process they went through for developing a system to collect data from robots during and before matches, and process this data for use after matches
The system for analyzing data must demonstrate a unique process in accounting for factors about the robot’s capabilities, consistency, and overall performance
The system to collect data is able to take information about uncommon occurrences, such as field/robot faults, penalties, and other uncommon events
A team can demonstrate how their data analysis can help them make decisions with strategy and alliance partners
What really comes to mind when I think of these criteria are the teams that build really cool match/pit scouting apps, and can use the data for all sorts of interesting applications, such as predicting match scores, cycle times, and ideal alliance selection picks.
This is obviously not something that is set in stone or has to exist next season (it’s a bit too late to add a new award this close to kickoff), but I think it could be a good way to recognize a subteam that generally doesn’t get much recognition outside of alliance selection and match strategy.
No comment on whether this is appropriate (it definitely could be), but one challenge is that FRC gives almost all awards at every competition. I would guess that at 98% of competitions, there is no team which really is doing “innovative” scouting and data analysis.
Maybe a better approach is to try to get this allowed under one of the other awards. “Innovation in Control” obviously does not fit, but another one?
The way I see it the occasional innovative and insightful data collection and/or analysis could fall under the judges award.
I have never been in a judges room, so I am not sure of how that judges award convention goes (I imagine it is just the team that deserves recognition that is left after all the other awards have been assigned)
Judges are normally a long ways away from the stands, but if in the pits you were to showcase live data flows, visuals, and how that all factors into strategy colb and teamwork I suspect that is a good case to be recognized by a judges award.
That’s why I addressed that argument, since I notice its been brought up in the past but I haven’t seen anyone mention the possibility of having an unlucky match schedule or not being able to rank top 8, which is especially common at champs since most robots don’t play against each other (even high tide ranked was ranked 17th in Hopper).
then industrial design = robot doesn’t break = good performance = wins = awards
or autonomous = more points = wins = awards
or excellence in engineering = good robot = wins = awards
or quality = good robot = wins = awards
Its not a valid argument that scouting leads to winning other awards, because we already have many awards that the same argument could be made
Go give your scouting system and/or analytics sales pitch to the judges during your pit interviews. Give them the hard sell. It could absolutely warrant a judge’s award.
But I don’t think there’s enough team’s doing enough interesting things with analytics or scouting to validate the award as a stand-alone award at each and every event this season.
It would be interesting to see what awards would make sense at a championship level. I’d argue that there are enough teams doing cool stuff with scouting to warrant an award at champs.
Right now I believe that they give the same awards at champs compared to a regional, but I don’t see why we can’t have different/more awards.
Full context on that quote please. I amAndyMark not necessarily saying that is my opinion, I amAndyMark just emphasizing the past angsty group-think on the matter in this echo chamber we call Chief Delphi.
If memory serves, that particular argument in the past was built around scouting not having any sort of direct measure in the same way as the other awards and logistical considerations. By the time most teams actually use scouting data/ scouting strategy it is alliance selection. The results of elims (where superior execution on scouting leading to the alliance over performance) have not happened yet. This is too late for judged awards - those decisions are already made around alliance selection time at events.
I would also like to point out that good scouting and data collection/ analysis are not necessarily correlated. We have talked at length on this forum about how massive data collection and scouting culture is poor. Yet a pair of motivated students scouting on vibes can easily out perform another team with an in house app, data analytics and 20 students dedicated to the task. What matters are the results, not the system.
I think FIRST wouldn’t add this award unless it was sponsored, just because it is harder to judge, there are fewer teams doing novel work, it is rewarded in alliance selection, and they have limited time with volunteer judges. I think they may be open to a sponsor providing criteria for an analytics award. For example, the Analytics Award sponsored by Tableau, would be something FIRST might work with despite some of the difficulties listed above. So if you can find and convince a global award sponsor, like now we have Dow sponsoring the Sustainability Award, that you could get scouting & analytics more recognition.
At the division level, same awards minus WFFA/Impact/DL. At the overall level, add Founder’s award and the full WFA/Impact (and sometimes honorable mentions)/DL.
The key part to these differences is that at Champs, they’re giving the highest level of the award, and additional awards aren’t aimed at teams.
If you want to win a scouting award, go to Beach Blitz (or another offseason that gives one). It’s not that it can’t be done, but it does need judging criteria, judges, and quite possibly a healthy dose of ability to figure out whether a team that uses GSheets or a team using Tableau is more worthy of the award if they get the same results.
I would be okay if they opened some of the “machine” award descriptions to allow for scouting systems to fall under them. For example, the innovation in control (or in my mind, “the programming award”) description could be modified to be less controls focused and more of a generic award for software.
Like someone else said, the judges award would allow for this as well.
I do think with some strategery, you could use scouting to boost your chances at awards:
“We revamped our scouting system starting with culture by doing XYZ and have seen ABC as a result” is a good add to Impact
“Not only is our robot quality but our scouting system is X accurate and has these redundancies” could work towards Quality
“We taught other teams how to scout at THING and X people were there. We are sharing our data with Y teams at the event” could count towards GP award
Yes. Yes. Yes. Can’t like this enough. I’ve said for awhile now scouting items is one of the things not addressed in the “awards” part of the competition and should be.
One other note on this award, and it’s kind of an important one.
Teams tend to stick with the same scouting system for years on end, once they pick one and develop it. It’s pretty rare for it to change annually unless someone is doing a refresh cycle (obviously the items being scouted change, but the core doesn’t really change that much).
So the question becomes, do you treat this Scouting Award like Impact, where you look at the past X seasons with an emphasis on this year/more recent, or do you treat it like robot-based awards where it’s just what the team did this year?
This doesn’t seem like a huge problem to me. There are teams who reuse CAD, code, and even parts of robots year after year. We would never say that a team should be ineligible for a design or programming award, just because they recycled components from previous seasons. Two major points:
You have to know how something works well enough to reuse it and make it competitive.
Award-worthy work, whether on the robot or in a scouting system, will have game-specific value that cannot be carried over year-to-year.
The question I’m asking is: If the award is for a system, and the system as a whole is reused year-to-year, how do we handle that?
This would be more akin to using a 2018 robot to play 2019 than to using parts of a 2022 swerve in 2023, IMO–modification is minimal, and the former is Right Out while the latter is Right, Good, Carry On (at least under certain common circumstances).
There’s actually a negative to this award existing, too, IMO. It’s pretty common around here for teams to share their scouting systems with other teams by posting them online with “here, take this”. Naturally systems get copied and tweaked. If there’s suddenly an award for “best scouting system” with whatever metrics, what happens to that flow of scouting systems? Me being a bit of a cynic, I’d say probably it gets cut in half pretty quick as teams that care about winning the award stop sharing while teams that prefer helping others continue.
I don’t really care one way or the other on long-term vs short-term, to be honest. Just that there is some sort of defined metric on how something that’s been around for years and still is used and updated might relate to something brand-new or something that’s copied from somewhere else and iterated/modified. (And I care even less what that metric is. I’m not the one who has to figure the award out.)
I’m challenging the premise that any system meritorious of anward could be reused year-to-year.
I’m not advocating that this awards should exist. I don’t know exactly how I feel. There are lots of valid arguments for and against. That said, I don’t think the recycling risk is any different than most other awards, and I don’t think that risk is very great with any awards now.
Personally, an innovation in scouting award seems overly specific compared to the other awards, even as a team that developed one this past year - that was hard. I think there’s space for another more general award that could apply to scouting, or some other application of innovative software not on the robot. But I think scouting remains on the list of things that have become common, but not quite universal enough to be accepted and acknowledged by FIRST as an officially endorsed part of the program.
It’s not that it can’t be done, but it does need judging criteria, judges, and quite possibly a healthy dose of ability to figure out whether a team that uses GSheets or a team using Tableau is more worthy of the award if they get the same results.