Scouting Metrics

Hey everyone!

I’m wondering how everyone is doing with their scouting systems this year. What kind of metrics have you decided upon? What format are you using? (paper, apps, excel, etc.) Overall, how are things shaping up for your scouting teams?

Personally, we (6081) are planning on using FRC Krawler this year. I’ve been very satisfied with the way that it is handling the data based on the metrics that we’ve chosen to use. (Shout out to 2052 for such an awesome system woohoo!)

As for metrics, we’ve locked down metrics for both autonomous and the endgame, but struggled a bit with coming up with ideas for tele-op. The students and I ultimately decided that solely keeping track of the number of cubes that a team can run wouldn’t be quite as effective, seeing as the necessary number of cubes in any given location has the potential to vary wildly from match to match. What we decided to do is keep track of where the cubes can be placed by any given team, and then give the team an efficiency rating for the tele-op portion. This should hopefully more or less operate as an estimated cycle time, without having to time each cube placed. HOWEVER, we realized that it may be worth tracking the number of cubes that a bot can place in the exchange, because it’s the only location where the opposing alliance won’t be placing cubes.

Overall, we’re quite satisfied with the way our testing up until now has turned out, and we’re hoping that it continues to prove successful!

I agree, the success rate of placing in a given area (including the exchange) is important. We’re behind on our scouting (like everything else…), but some other things I’d want to track:

  • Fouls
  • Robot deaths/no-shows
  • Blown pickups (where they grab, miss or drop it quickly, and have to stop to grab again). I suspect this will make it clearer when a team has a weak robot versus a weak strategy.
  • Defensive moves (painted broadly and subjectively) Over eight or ten matches, you’ll see some standouts.
  • If a fast climb is significant to you, maybe some stat for “time spent screwing around in the platform zone”.

KISS…

Cube acquire time scoring placement time (secs)
climb? Y/N
auto? Y/N

In the end that covers 90% the other 10% driver smarts
Be careful of scouting information overload.

I would argue this doesn’t give you enough information to determine the strategies that teams are performing. For example, there is a difference between putting a cube on the scale or on the switch. Having just a cube cycle time doesn’t tell me where the cubes are being placed. I agree with the KISS principle, but there needs to be a balance between getting the data that helps a team make calculated decisions while keeping the scouting simple.

With so many possible scenarios, I agree that this is a hard game to scout in Teleop. That said, a good drive team will be making things happen on the field so I believe if a team is playing offense, then the total # of cubes placed somewhere is important. The circumstances may dictate a change in tactics, but a robot should be doing something at all times IMO if it isn’t having technical difficulties.

We plan to use tablets to scout and dump that data along with the FRC event data into excel and generate reports.

This year I decided that we’ll be using pen and paper, because the wifi at our events usually sucks, and a lot of our team doesn’t always have access to cell data. We have a spreadsheet tracking climbing, cubes, if it delivers properly in auto, defense, and the lovely “other comments” column.

I totally agree. This year it was HARD to determine what metrics will actually be valuable. In addition, CD wasn’t super helpful this year, because the bulk of responses sound like this. (directed at no team in particular)

Qualitative data is important, we have been doing qualitative for years, and we love this game. And it’s gonna be super important, so we’re gonna record qualitative data.

great… I like qualitative data as well, but that doesn’t give me any specifics to go off of.

So after weeks of thinking/discussion, we finally decided on having individual scouts watch one robot and scout…

AUTON:

Autoline Crossed(y/n)

of attempts to place cubes on Switch

of successfully scored cubes on Switch

of attempts to place cubes on Scale

of successfully scored cubes on Scale

(scored/attempts will give us success rate, which is helpful to see improvement due to driver practice)

TELEOP:

of attempts to place cubes on Switch

of successfully scored cubes on Switch

of attempts to place cubes on Scale

of successfully scored cubes on Scale

of cubes placed in exchange

Endgame Position(Not scoring, Parked, Climbing)

POST MATCH:

Whether they were disabled or not at any point

Comments(notes on driver skill, speed, haiku about scouting)