Best Ways to Scout

[Citrus Dad]

Originally Posted by Michael_Rood

As a first year scouting captain, I have had trouble finding a reliable way to scout. In the years before I became scouting captain, my team has been using paper scouting. This year we made the switch over to electronic scouting. We used Google Forms to create a scouting sheet that wee could compile into an Excel sheet. The one problem is, it requires a reliable wifi connection to work. I was wondering if there were other ways to scout electronically without having to rely on wifi. I have been talking with one of my mentors and have come up with some ideas. 1. Having a portable server that we can wire the phones into (would require making an app that doesn't need wifi). 2. Having the phones connected to a laptop via Bluetooth. If someone could help me build an app (in LabVIEW preferably) that could connect to a server (would also need to find a server that would work with the app) I would greatly ly appreciate it.

We've posted our scouting white paper in at least 3 threads this year alone. Search for my posts if need be.

We use a system that combines Bluetooth and cell connections and avoid WiFi. We scout on tablets.

[Abhishek R]

Paper to Excel is a pretty reliable, time-tested method.

[jajabinx124]

Originally Posted by Citrus Dad

We've successfully set up a scouting system that can use relatively inexperienced students to keep quantitative counts. We have to work to find the other 2 superscouts who make qualitative judgements. Comparing our system to what we see at regionals and even Champs, we believe having a more rigorous system if more fruitful.

We essentially have the same type of system. We have inexperienced to experienced members being our regular stand scouts keeping quantitative counts, but in our scouting application there is a comments section where our scouts summarize the match in words so that's why we don't need super scouts. Although, some experienced scouts still take notes and as the strategist whenever I watch matches I take notes, so in a way we have super scouting. It's worked pretty well for us so far, but we still have some refining to do there in preparation for champs.

Originally Posted by Abhishek R

Paper to Excel is a pretty reliable, time-tested method.

I'll agree with this. Even though my team uses an android scouting application, paper to scouting (or even laminated sheets, I recommend laminated sheets) can be SUPER reliable as well- try and make a good paper/excel scouting system if your team can't muster a good electronic scouting system IMO. Paper can get annoying if not properly organized, but if properly organized, it can works wonders.

[Boltman]

Originally Posted by jajabinx124

We essentially have the same type of system. We have inexperienced to experienced members being our regular stand scouts keeping quantitative counts, but in our scouting application there is a comments section where our scouts summarize the match in words so that's why we don't need super scouts. Although, some experienced scouts still take notes and as the strategist whenever I watch matches I take notes, so in a way we have super scouting. It's worked pretty well for us so far, but we still have some refining to do there in preparation for champs.



I'll agree with this. Even though my team uses an android scouting application, paper to scouting (or even laminated sheets, I recommend laminated sheets) can be SUPER reliable as well- try and make a good paper/excel scouting system if your team can't muster a good electronic scouting system IMO. Paper can get annoying if not properly organized, but if properly organized, it can works wonders.

I write super sloppy BUT paper can tell the story..I use highlighter colors to explain the story and Excel with colors so others can read it. Boils down to in essence are they a help or hindrance and are they reliable and consistent?

KISS is a great way to go.

[oliviat3357]

I can speak pretty highly of electronic scouting. The first two years I was on our team, we used paper scouting and it was so slow and messy. This year, someone created an app for Windows computers to run on c#, and it's great because we can always be making changes between competitions. Basically, 6 people watch the match and fill out the form. After a few matches, someone collects the data via USB or network and it can generate sheets for matches. This is nice, because it allows you to compare red and blue alliance data, make sorted pick lists based on shooting, outer works, etc.

It's the most organized I've ever felt for our team. NEVER let scouters skip the comments section, though. Not only will it be the most useful information, but you'll also find some pretty funny responses when you look back.

[Citrus Dad]

Originally Posted by BenjiSG

We made the decision this year to just stick with paper scouting. Electronic scouting is nifty and all, but the inconveniences of trying to get reliable WiFi and access to power are too much to make it actually worth it for us. Plus, lost paper is far preferable to lost electronics, and we don't see a great deal to gain from going digital.

I'll make 2 points. The first is more basic--you can use Bluetooth and cellphone data to transmit which avoids WiFi. See our scouting whitepaper. And we use tablets which can get through an entire day of scouting-you only need to charge at night. Plus the system is fairly reliable--at least as much as paper when those can get scattered. The final one is that it is much easier to communicate with the drive team in real time with an electronic system. (Scouting is about much more than draft lists.)

My second point is about philosophy. The point of FRC is not competition and winning; it's about education and training (which is fun in a sports setting). Having your students create an electronic scouting system is another educational opportunity. We have many students programming apps, servers and other devices. They learn about math and statistics. This has been one of the best educational opportunities for our team.

[Boltman]

Originally Posted by Citrus Dad

I'll make 2 points. The first is more basic--you can use Bluetooth and cellphone data to transmit which avoids WiFi. See our scouting whitepaper. And we use tablets which can get through an entire day of scouting-you only need to charge at night. Plus the system is fairly reliable--at least as much as paper when those can get scattered. The final one is that it is much easier to communicate with the drive team in real time with an electronic system. (Scouting is about much more than draft lists.)

My second point is about philosophy. The point of FRC is not competition and winning; it's about education and training (which is fun in a sports setting). Having your students create an electronic scouting system is another educational opportunity. We have many students programming apps, servers and other devices. They learn about math and statistics. This has been one of the best educational opportunities for our team.

Agree, electronic collection offers another avenue of learning. In a way though today's kids are already overloaded with electronics and there is nothing wrong with teaching them to make observations, discuss strategies pre-match and learning to evaluate the entire team while taking relevant notes. In a way, simply getting away from electronics to do everything itself is a learning experience for them. One problem with today society is much of it believes an app solves everything. It doesn't. Many of these kids were born with various electronics in hand and lack the ability to evaluate manually and look beyond stats and at other clues not mentioned much on CD.

I see value in both aspects and one is not better than the other. So when the OP asks "Best way to scout" I don't take that as "best way to learn about statistics and app development" necessarily.

We all have ways that we think works and probably tweak it yearly... they are all learning opportunities. I go back to sample size and number of matches being an issue with heavy reliance on quantitative scouting. Most teams only play about 22 minutes per regional in qualifications and only against and with certain teams so I find that numbers are easily skewed base on pairings in any single event. I try to cut down data points... instead of what they can do what cant they do, an where do they prefer to start are a few observations we track.

In the end I use both methods one to validate the other and do look for learning opportunities for the kids. I think there is certainly room in app development to track things that aren't tracked in the apps I have tested. Many seem to track too much and at the same time not enough IMO. Cutting down the noise is good too and focusing a a few key areas seems to do fine. I do love the stats many post here kudos to them.

[Squillo]

We had great success this year using google forms to collect data. The kids input data from each match on their phones, and it went to a spreadsheet. I had a computer in the pit, with a USB modem to get internet without wifi. I wrote an analysis spreadsheet, that I used to compile and analyze the data, and create reports (transcribed by hand ontompaper from the spreadsheet data, but we may use electronic delivery next year) for the drive team. We had data for which defenses each bot could cross, which they'd had trouble with, high and low shot numbers and percentage success, points scored per match (auto, breach, goal, endgame, and total), speed/agility, defense, and trends over time.

At our regional, we collected over 450 individual reports (each report is about one bot in one match), had great info about each team by Friday night (and better by the end of quals), and used it to help our alliance captain make an awesome (and overlooked) 2d pick, as well as to plan strategy and choose defenses for every match. With a small team, with drivers who were also repairing the robot and working on the software, and really tight turnaround times between some matches, it was great to have the scouting data automatically "beamed" right to the pit.

Next year it will be even better, and if we can get a few more students to join the team, I will hopefully find one who wants to take it over.

I also got an awesome "8x" Mophie charger brick from Verizon, that kept everyone powered up all day.

[Michael Hill]

We have been doing electronic scouting since 2013. We set up a Bluetooth network with tablets (one tablet per robot). We also keep a few tablets charging at all times and one in the pits. There are ups and downs. The ups are obviously realtime data (and a heck of a lot of it). The downside is the human element. We are a small team, so wherever we go we partner up with other teams to help scout. The downside is you have to train people as well as deal with mistakes on input. Someone had likely made a mistake when they said we went under the low bar in autonomous (we can't fit). Additionally, several high goal shots are missed in the first match we shot high (kind of disappointing). However, for the most part, we get good data. Here is our app: http://innovators3138.org/Fires16/CS/#!/

Additionally, we leave the data open to all who wants it. It's the analysis of the data that we keep private (though admittedly, there's not much to it).

[Citrus Dad]

Originally Posted by Boltman

Agree, electronic collection offers another avenue of learning. In a way though today's kids are already overloaded with electronics and there is nothing wrong with teaching them to make observations, discuss strategies pre-match and learning to evaluate the entire team while taking relevant notes. In a way, simply getting away from electronics to do everything itself is a learning experience for them. One problem with today society is much of it believes an app solves everything. It doesn't. Many of these kids were born with various electronics in hand and lack the ability to evaluate manually and look beyond stats and at other clues not mentioned much on CD.

I see value in both aspects and one is not better than the other. So when the OP asks "Best way to scout" I don't take that as "best way to learn about statistics and app development" necessarily.

We all have ways that we think works and probably tweak it yearly... they are all learning opportunities. I go back to sample size and number of matches being an issue with heavy reliance on quantitative scouting. Most teams only play about 22 minutes per regional in qualifications and only against and with certain teams so I find that numbers are easily skewed base on pairings in any single event. I try to cut down data points... instead of what they can do what cant they do, an where do they prefer to start are a few observations we track.

In the end I use both methods one to validate the other and do look for learning opportunities for the kids. I think there is certainly room in app development to track things that aren't tracked in the apps I have tested. Many seem to track too much and at the same time not enough IMO. Cutting down the noise is good too and focusing a a few key areas seems to do fine. I do love the stats many post here kudos to them.

I think the original question should be redirected. (That's often the role of a mentor addressing a student.) My point is on the "best way to scout" is to keep in mind the overall mission of FIRST which is to focus on the educational aspect. So in looking at scouting, we should look beyond competition but to the overall educational mission of FRC. Whether a scouting system is truly successful is really secondary to what students learn in the process of developing and implementing the scouting system. That's why we provide white papers on developing the systems, not a turnkey version of the system itself. We want other teams to build their own systems.

We've found that the students and younger mentors learn how to better mesh data analysis and strategic observation. The draft lists we generate are only a starting point--it helps separate the very best, the middle and the low end, and then we have to rerank the middle in particular with our own observations. It's an education process in the limits of strict data analysis.

The fact is that this is really on the only venue in high school where students can directly learn about how to use the convergence of data and direct observation. Most social science classes convey the observational part through histories and theories, and the stats classes teach the mechanics of data analysis. I don't think a team should miss the opportunity of providing yet another project-based learning experience that brings together several different educational threads. This is what using "Big Data" is about. Decision science and analysis are growing majors in college and FRC grads can be well positioned to perform well in those programs.

So I wouldn't want to leave scouting solely to an exercise in observational judgement. There are other many other venues for that type of experience, but FRC provides a truly unique opportunity that no team should pass up.

[bennettj800]

Originally Posted by Squillo

We had great success this year using google forms to collect data. The kids input data from each match on their phones, and it went to a spreadsheet. I had a computer in the pit, with a USB modem to get internet without wifi. I wrote an analysis spreadsheet, that I used to compile and analyze the data, and create reports (transcribed by hand ontompaper from the spreadsheet data, but we may use electronic delivery next year) for the drive team. We had data for which defenses each bot could cross, which they'd had trouble with, high and low shot numbers and percentage success, points scored per match (auto, breach, goal, endgame, and total), speed/agility, defense, and trends over time.

At our regional, we collected over 450 individual reports (each report is about one bot in one match), had great info about each team by Friday night (and better by the end of quals), and used it to help our alliance captain make an awesome (and overlooked) 2d pick, as well as to plan strategy and choose defenses for every match. With a small team, with drivers who were also repairing the robot and working on the software, and really tight turnaround times between some matches, it was great to have the scouting data automatically "beamed" right to the pit.

Next year it will be even better, and if we can get a few more students to join the team, I will hopefully find one who wants to take it over.

I also got an awesome "8x" Mophie charger brick from Verizon, that kept everyone powered up all day.

The problem here is that not everyone has the kind of data required to do that... so when you just send kids out with their phones and say to use this online form you made, they wont always have access to that form! We tried that last year and people got really mad at me... like, *really upset* kinda mad... so just keep the internet out of play when you roam the pits, unless there is some sort of wifi (which there should not be!)

[Squillo]

I was surprised at how little data it used. I signed up for a few extra GB, in advance of our two regionals, but found that even using my phone AND my computer (the USB modem also uses 3G/4G data, it's another device on our share plan) for three solid days used less than 0.5 GB of data.

The one student who had no available data filled out paper forms and others took turns putting the data into the online form. It really didn't take very long and they didn't mind.

We now have a pro data analyst who has been doing some analysis of the FIRST data feed from all of the competitions, and next year he is going to work with the kids to do more different kinds of analysis, and refine our scouting processes. He and I were comparing our outcomes after the Hawaii Regional, and we both agreed that BOTH kinds of data - the purely score-based, quantitative stuff from the feed, which can provide OPR stats and the like, AND the more qualitative stuff that we got from "eyes on the field," along with more specific "This Robot Can/Can't Do This" (which cannot be gained from the match data alone, since it is alliance-based, not individual team-based) are necessary to get the whole picture. This is great stuff for the students to be learning and we're going to do more next year. For this year, it was a good first step to show them just what can be obtained and gained from having the data, and how it can be compiled and analyzed to produce useful output.

[Citrus Dad]

Originally Posted by bennettj800

The problem here is that not everyone has the kind of data required to do that... so when you just send kids out with their phones and say to use this online form you made, they wont always have access to that form! We tried that last year and people got really mad at me... like, *really upset* kinda mad... so just keep the internet out of play when you roam the pits, unless there is some sort of wifi (which there should not be!)

One solution is to beam via Bluetooth to a central phone or tablet that then uploads the data. That avoids using cell data. But the truth is that text uses almost no data. We buy a monthly cell plan to transmit data, and we've never gotten close the ceiling.

[evanperryg]

Originally Posted by bennettj800

The problem here is that not everyone has the kind of data required to do that... so when you just send kids out with their phones and say to use this online form you made, they wont always have access to that form! We tried that last year and people got really mad at me... like, *really upset* kinda mad... so just keep the internet out of play when you roam the pits, unless there is some sort of wifi (which there should not be!)

There are ample ways around this. Saving data to text files which are then copied off of each scout's device is the easiest way, if EXTREMELY inefficient. We use an android app that interfaces with an AWS database. Scouting a thousand matches would use less than 1Mb, and our scouters know that they should check how much data is left on their plan, and if theyre running out, to use the "save to text file" mode on our scouting app instead.

On that topic... GearScout!
GearScout is THE tried and true scouting app for both iPhone and Android devices. (iPhone version is now available, sorry about the delay... apple takes their time) 2 teams on Einstein last season were GearScout users, and GearScout has been the app of choice for numerous regional winners and finalists. We provide flexible save options for teams wishing to take advantage of the simplicity of CSV files, and offer the ability to save data to the Gear it Forward Amazon DynamoDB database, and we also give the option to have each line of data send itself over text to a lead scout. The app automatically calculates points contributed, and offers the ability to switch into "notes mode" for qualitative scouters. Database data is secure; your team is the only one that can access your data. Search "FRC GearScout" on Google Play or the iPhone App Store. If you check my media posts, you can find a barebones spreadsheet for importing the CSV files and viewing them. If you have any questions, or you're using our app this season, PM me! I'm always happy to help resolve issues with the app, and we love to hear the success stories of teams who use our system.

[BenjiSG]

Originally Posted by Citrus Dad

I'll make 2 points. The first is more basic--you can use Bluetooth and cellphone data to transmit which avoids WiFi. See our scouting whitepaper. And we use tablets which can get through an entire day of scouting-you only need to charge at night. Plus the system is fairly reliable--at least as much as paper when those can get scattered. The final one is that it is much easier to communicate with the drive team in real time with an electronic system. (Scouting is about much more than draft lists.)

My second point is about philosophy. The point of FRC is not competition and winning; it's about education and training (which is fun in a sports setting). Having your students create an electronic scouting system is another educational opportunity. We have many students programming apps, servers and other devices. They learn about math and statistics. This has been one of the best educational opportunities for our team.

At most events I've been to, cell reception has often been spotty at best. We also investigated Bluetooth, but were put off by price and hassle.

Our core scouting team is relatively small- generally, there are three people who actually dedicated and the rest are simply members we get to scout matches while in the stands at the competition. There's definitely value towards devising your own system and learning all of the necessary skills involved, but since our dedicated scouting leads all have other demanding jobs which are also requiring of their time, paper scouting seems to give us the best efficiency with the least effort required. It may sound lazy, but it's really just a solution to make sure that we can show up with effective scouting without tearing people from other projects.

A larger, even more dedicated scouting team could likely complete such a project and get through all of the workarounds, but it honestly seems to be for a different goal from simply scouting in the "best" way possible- that is, the way which can accurately inform selections while being relatively low maintenance. Scouting for the learning experience is excellent for a team with members dedicated to putting vast amounts of time into the intricacies of processing data and utilizing statistics, but paper scouting gets the job done and gets it done well.