Best Ways to Scout
 

As a first year scouting captain, I have had trouble finding a reliable way to scout. In the years before I became scouting captain, my team has been using paper scouting. This year we made the switch over to electronic scouting. We used Google Forms to create a scouting sheet that wee could compile into an Excel sheet. The one problem is, it requires a reliable wifi connection to work. I was wondering if there were other ways to scout electronically without having to rely on wifi. I have been talking with one of my mentors and have come up with some ideas. 1. Having a portable server that we can wire the phones into (would require making an app that doesn't need wifi). 2. Having the phones connected to a laptop via Bluetooth. If someone could help me build an app (in LabVIEW preferably) that could connect to a server (would also need to find a server that would work with the app) I would greatly ly appreciate it.

Thank You

Best Ways to Scout
 

We made the decision this year to just stick with paper scouting. Electronic scouting is nifty and all, but the inconveniences of trying to get reliable WiFi and access to power are too much to make it actually worth it for us. Plus, lost paper is far preferable to lost electronics, and we don't see a great deal to gain from going digital.



Frankly, just printing out excel sheets, allowing them to be filled in by hand, and compiling all useful information digitally after Day 1 proves to be far easier and seemingly just as effective.



If you want to move to electronic scouting, by all means, but don't forget that the paper strategy remains not only an option, but in some ways a more preferable one.

Re: Best Ways to Scout
 

Try looking through the myriad of threads on the issue in the scouting forum. There are a lot of really good ideas and tips on this issue.

Just click on the forums thread in the orange bar on top. Then go to the scouting sub-forum. Its all there for you.

Re: Best Ways to Scout
 

Best way to scout is "eyes on bots", excel sheets , pen and paper note taking. I use a cheap thin spiral binder and highlighters . My awesome scouts also take copious notes. I started the scouting department from scratch last year. Scouting is equal to engineering and drive team IMO and the entire team agrees with me they love it. Not only for winning games but building alliances to have the best chance at a win.

A field of 60 with 10-12 games is not indicative of a high level of need for data analysis and is highly susceptible to schedule pairings more than raw data from stats. Eyes first, notes second and data on first night to see if there are any you missed. Usually not the case...might be one or two but then you watch on day 2 and go "hmm wonder how they got that stat?" eyes don't lie like stats do.

I don't get alliance captains that don't already know their second picks instantly of a pick list, other than they must not scout at all and ultimately they exit in QF.

Re: Best Ways to Scout
 

This year we are using an html form we created on a local network.

We have a local server that everyone connects to with Ethernet cords. All the data gets pushed to an SQL database on that server. It looks like a mess, but it works really well.

Re: Best Ways to Scout
 

I didn't see this (so much for "eyes first") but I heard from several people that our robot accidentally scored a high goal once during quals. We're not even really high goal capable but apparently during one match we were driving forward and somehow a boulder got launched at just the right angle that it arced into the goal. Total fluke. I'm sure a bunch of teams were looking over their scouting data later going "wait, what? That's not right...!"

Re: Best Ways to Scout
 

Our team creates a scouting application to use on android tablets. This way we can write our own analysis program, to find what information we need about any team.

Re: Best Ways to Scout
 

What devices are you using to scout?

Re: Best Ways to Scout
 

If you want to maintain the electronic aspect of your scouting and have the phones/tablets to do it, I suggest SuperScouter. It's schema based, so it's customizable and I found the original thread to be very helpful.

http://www.chiefdelphi.com/forums/sh...t=superscouter

While I couldn't get the QR scanner to work, I'm planning on using micro-USB to Ethernet and lightning cable to Ethernet to sync everyone's phones to my laptop and then inputting that data into Excel. Then I input the data I want into Tableau, which is a visualization software free to FRC teams!

http://www.tableau.com/first-robotics

Re: Best Ways to Scout
 

I have written my own database and hope it helps anyone looking for one.

i think a hybrid between paper and PC is the best way to go, with a projector for the meeting that night so everyone attending can see what the hay hay is going on.

[|HERE|] is that database
and if that ↑ didnt embed correctly, the link is here:
https://drive.google.com/open?id=0B4...FpFTC04d2doYjg

I STRONGLY suggest using this database (youll appreciate all the formatting)

I aso STRONGLY suggest using the sheets that come with it. If you do not you will lose more information than you think, because not all will transfer into the database, and not all of it will be available for the database... that being said, USE THESE PLEASE

Other than that, I just really dont wanna waste this exvel spreadsheet... its really nice... i hope you use it... I really really hope...

Re: Best Ways to Scout
 

I could agree with the "eyes over stats" idea IF you have experienced scouts* that know what to look for and IF you have enough experienced scouts to get eyes on a majority of matches for every team. You realistically need at least four experienced scouts to literally watch 90% of matches. Since most FIRST teams do not have these resources, putting together a stats-based system (where any freshman can contribute) is best for most teams.

* By experienced scout, I mean a scout that can remember basic information about nearly all teams and put teams in tiers or rough picklist. In the roughly 10 years I've scouted, there are probably 3-5 students who I'd consider capable of doing this for an entire 60+ team event.

Re: Best Ways to Scout
 

My team uses an electronic reporting form, but we use phones and tablets that connect directly to the internet. We also value paper scouting. One of our mentors always takes paper notes on each robot's performance across one or two matches and we rearrange these cards when making our pick list Friday night. I also want to point out the advantages of making a pick list of Friday--it may seem smart not to make these kinds of decisions until just before alliance selections, but you will be pressed for time at this point and it's best to have an idea of the order of teams that you can rearrange when teams either increase or decrease their level of performance.

Re: Best Ways to Scout
 

STORM Robotics also has created an Android application! We are particularly proud of this!

More info and the link to the Play Store is in this following thread.

http://www.chiefdelphi.com/forums/sh...hreadid=142649

It is easy to use, very thorough, and completely offline, utilizing QR codes and outputting .csv files which can be opened in excel for easy management and analysis. There is a User app for data entry, and then a Master app for collection and creation of the CSV files. I hope you try it out! We, along with several other teams, have had great success using the app at our respective competitions.

Re: Best Ways to Scout
 

You can always use google sheets offline on devices. Simply set up an excel form on google sheets, and people can edit the excel on their device. To compile simply connect the phones to a computer (android only) or go outside at the end of the day and send it with 4g. I have some google sheets forms handy if you want them.

Re: Best Ways to Scout
 

We've successfully set up a scouting system that can use relatively inexperienced students to keep quantitative counts. We have to work to find the other 2 superscouts who make qualitative judgements. Comparing our system to what we see at regionals and even Champs, we believe having a more rigorous system if more fruitful. For example, we were able to get our 10th overall pick (even accounting for alliance captains) at Sacramento as our 2nd pick. 971 and us use a similar rigorous system and our draft lists agreed quite well. (5274 should thank 971 for jumping them up a spot on the list.)

We know who we DON'T want by the end of the second day, but our mid-field list where our 2nd picks reside is much more cluttered and subject to revision. (We've gotten by the QFs every competition since 2013...;) )

Re: Best Ways to Scout
 

We've posted our scouting white paper in at least 3 threads this year alone. Search for my posts if need be.

We use a system that combines Bluetooth and cell connections and avoid WiFi. We scout on tablets.

Re: Best Ways to Scout
 

Paper to Excel is a pretty reliable, time-tested method.

Re: Best Ways to Scout
 

We essentially have the same type of system. We have inexperienced to experienced members being our regular stand scouts keeping quantitative counts, but in our scouting application there is a comments section where our scouts summarize the match in words so that's why we don't need super scouts. Although, some experienced scouts still take notes and as the strategist whenever I watch matches I take notes, so in a way we have super scouting. It's worked pretty well for us so far, but we still have some refining to do there in preparation for champs.

I'll agree with this. Even though my team uses an android scouting application, paper to scouting (or even laminated sheets, I recommend laminated sheets) can be SUPER reliable as well- try and make a good paper/excel scouting system if your team can't muster a good electronic scouting system IMO. Paper can get annoying if not properly organized, but if properly organized, it can works wonders.

Re: Best Ways to Scout
 

I write super sloppy BUT paper can tell the story..I use highlighter colors to explain the story and Excel with colors so others can read it. Boils down to in essence are they a help or hindrance and are they reliable and consistent?

KISS is a great way to go.

Re: Best Ways to Scout
 

I can speak pretty highly of electronic scouting. The first two years I was on our team, we used paper scouting and it was so slow and messy. This year, someone created an app for Windows computers to run on c#, and it's great because we can always be making changes between competitions. Basically, 6 people watch the match and fill out the form. After a few matches, someone collects the data via USB or network and it can generate sheets for matches. This is nice, because it allows you to compare red and blue alliance data, make sorted pick lists based on shooting, outer works, etc.

It's the most organized I've ever felt for our team. NEVER let scouters skip the comments section, though. Not only will it be the most useful information, but you'll also find some pretty funny responses when you look back.

Re: Best Ways to Scout
 

I'll make 2 points. The first is more basic--you can use Bluetooth and cellphone data to transmit which avoids WiFi. See our scouting whitepaper. And we use tablets which can get through an entire day of scouting-you only need to charge at night. Plus the system is fairly reliable--at least as much as paper when those can get scattered. The final one is that it is much easier to communicate with the drive team in real time with an electronic system. (Scouting is about much more than draft lists.)

My second point is about philosophy. The point of FRC is not competition and winning; it's about education and training (which is fun in a sports setting). Having your students create an electronic scouting system is another educational opportunity. We have many students programming apps, servers and other devices. They learn about math and statistics. This has been one of the best educational opportunities for our team.

Re: Best Ways to Scout
 

Agree, electronic collection offers another avenue of learning. In a way though today's kids are already overloaded with electronics and there is nothing wrong with teaching them to make observations, discuss strategies pre-match and learning to evaluate the entire team while taking relevant notes. In a way, simply getting away from electronics to do everything itself is a learning experience for them. One problem with today society is much of it believes an app solves everything. It doesn't. Many of these kids were born with various electronics in hand and lack the ability to evaluate manually and look beyond stats and at other clues not mentioned much on CD.

I see value in both aspects and one is not better than the other. So when the OP asks "Best way to scout" I don't take that as "best way to learn about statistics and app development" necessarily.

We all have ways that we think works and probably tweak it yearly... they are all learning opportunities. I go back to sample size and number of matches being an issue with heavy reliance on quantitative scouting. Most teams only play about 22 minutes per regional in qualifications and only against and with certain teams so I find that numbers are easily skewed base on pairings in any single event. I try to cut down data points... instead of what they can do what cant they do, an where do they prefer to start are a few observations we track.

In the end I use both methods one to validate the other and do look for learning opportunities for the kids. I think there is certainly room in app development to track things that aren't tracked in the apps I have tested. Many seem to track too much and at the same time not enough IMO. Cutting down the noise is good too and focusing a a few key areas seems to do fine. I do love the stats many post here kudos to them.

Re: Best Ways to Scout
 

We had great success this year using google forms to collect data. The kids input data from each match on their phones, and it went to a spreadsheet. I had a computer in the pit, with a USB modem to get internet without wifi. I wrote an analysis spreadsheet, that I used to compile and analyze the data, and create reports (transcribed by hand ontompaper from the spreadsheet data, but we may use electronic delivery next year) for the drive team. We had data for which defenses each bot could cross, which they'd had trouble with, high and low shot numbers and percentage success, points scored per match (auto, breach, goal, endgame, and total), speed/agility, defense, and trends over time.

At our regional, we collected over 450 individual reports (each report is about one bot in one match), had great info about each team by Friday night (and better by the end of quals), and used it to help our alliance captain make an awesome (and overlooked) 2d pick, as well as to plan strategy and choose defenses for every match. With a small team, with drivers who were also repairing the robot and working on the software, and really tight turnaround times between some matches, it was great to have the scouting data automatically "beamed" right to the pit.

Next year it will be even better, and if we can get a few more students to join the team, I will hopefully find one who wants to take it over.

I also got an awesome "8x" Mophie charger brick from Verizon, that kept everyone powered up all day.

Re: Best Ways to Scout
 

We have been doing electronic scouting since 2013. We set up a Bluetooth network with tablets (one tablet per robot). We also keep a few tablets charging at all times and one in the pits. There are ups and downs. The ups are obviously realtime data (and a heck of a lot of it). The downside is the human element. We are a small team, so wherever we go we partner up with other teams to help scout. The downside is you have to train people as well as deal with mistakes on input. Someone had likely made a mistake when they said we went under the low bar in autonomous (we can't fit). Additionally, several high goal shots are missed in the first match we shot high (kind of disappointing). However, for the most part, we get good data. Here is our app: http://innovators3138.org/Fires16/CS/#!/

Additionally, we leave the data open to all who wants it. It's the analysis of the data that we keep private (though admittedly, there's not much to it).

Re: Best Ways to Scout
 

I think the original question should be redirected. (That's often the role of a mentor addressing a student.) My point is on the "best way to scout" is to keep in mind the overall mission of FIRST which is to focus on the educational aspect. So in looking at scouting, we should look beyond competition but to the overall educational mission of FRC. Whether a scouting system is truly successful is really secondary to what students learn in the process of developing and implementing the scouting system. That's why we provide white papers on developing the systems, not a turnkey version of the system itself. We want other teams to build their own systems.

We've found that the students and younger mentors learn how to better mesh data analysis and strategic observation. The draft lists we generate are only a starting point--it helps separate the very best, the middle and the low end, and then we have to rerank the middle in particular with our own observations. It's an education process in the limits of strict data analysis.

The fact is that this is really on the only venue in high school where students can directly learn about how to use the convergence of data and direct observation. Most social science classes convey the observational part through histories and theories, and the stats classes teach the mechanics of data analysis. I don't think a team should miss the opportunity of providing yet another project-based learning experience that brings together several different educational threads. This is what using "Big Data" is about. Decision science and analysis are growing majors in college and FRC grads can be well positioned to perform well in those programs.

So I wouldn't want to leave scouting solely to an exercise in observational judgement. There are other many other venues for that type of experience, but FRC provides a truly unique opportunity that no team should pass up.

Re: Best Ways to Scout
 

The problem here is that not everyone has the kind of data required to do that... so when you just send kids out with their phones and say to use this online form you made, they wont always have access to that form! We tried that last year and people got really mad at me... like, *really upset* kinda mad... so just keep the internet out of play when you roam the pits, unless there is some sort of wifi (which there should not be!)

Re: Best Ways to Scout
 

I was surprised at how little data it used. I signed up for a few extra GB, in advance of our two regionals, but found that even using my phone AND my computer (the USB modem also uses 3G/4G data, it's another device on our share plan) for three solid days used less than 0.5 GB of data.

The one student who had no available data filled out paper forms and others took turns putting the data into the online form. It really didn't take very long and they didn't mind.

We now have a pro data analyst who has been doing some analysis of the FIRST data feed from all of the competitions, and next year he is going to work with the kids to do more different kinds of analysis, and refine our scouting processes. He and I were comparing our outcomes after the Hawaii Regional, and we both agreed that BOTH kinds of data - the purely score-based, quantitative stuff from the feed, which can provide OPR stats and the like, AND the more qualitative stuff that we got from "eyes on the field," along with more specific "This Robot Can/Can't Do This" (which cannot be gained from the match data alone, since it is alliance-based, not individual team-based) are necessary to get the whole picture. This is great stuff for the students to be learning and we're going to do more next year. For this year, it was a good first step to show them just what can be obtained and gained from having the data, and how it can be compiled and analyzed to produce useful output.

Re: Best Ways to Scout
 

One solution is to beam via Bluetooth to a central phone or tablet that then uploads the data. That avoids using cell data. But the truth is that text uses almost no data. We buy a monthly cell plan to transmit data, and we've never gotten close the ceiling.

Re: Best Ways to Scout
 

There are ample ways around this. Saving data to text files which are then copied off of each scout's device is the easiest way, if EXTREMELY inefficient. We use an android app that interfaces with an AWS database. Scouting a thousand matches would use less than 1Mb, and our scouters know that they should check how much data is left on their plan, and if theyre running out, to use the "save to text file" mode on our scouting app instead.

On that topic... GearScout!
GearScout is THE tried and true scouting app for both iPhone and Android devices. (iPhone version is now available, sorry about the delay... apple takes their time) 2 teams on Einstein last season were GearScout users, and GearScout has been the app of choice for numerous regional winners and finalists. We provide flexible save options for teams wishing to take advantage of the simplicity of CSV files, and offer the ability to save data to the Gear it Forward Amazon DynamoDB database, and we also give the option to have each line of data send itself over text to a lead scout. The app automatically calculates points contributed, and offers the ability to switch into "notes mode" for qualitative scouters. Database data is secure; your team is the only one that can access your data. Search "FRC GearScout" on Google Play or the iPhone App Store. If you check my media posts, you can find a barebones spreadsheet for importing the CSV files and viewing them. If you have any questions, or you're using our app this season, PM me! I'm always happy to help resolve issues with the app, and we love to hear the success stories of teams who use our system.

Re: Best Ways to Scout
 

At most events I've been to, cell reception has often been spotty at best. We also investigated Bluetooth, but were put off by price and hassle.

Our core scouting team is relatively small- generally, there are three people who actually dedicated and the rest are simply members we get to scout matches while in the stands at the competition. There's definitely value towards devising your own system and learning all of the necessary skills involved, but since our dedicated scouting leads all have other demanding jobs which are also requiring of their time, paper scouting seems to give us the best efficiency with the least effort required. It may sound lazy, but it's really just a solution to make sure that we can show up with effective scouting without tearing people from other projects.

A larger, even more dedicated scouting team could likely complete such a project and get through all of the workarounds, but it honestly seems to be for a different goal from simply scouting in the "best" way possible- that is, the way which can accurately inform selections while being relatively low maintenance. Scouting for the learning experience is excellent for a team with members dedicated to putting vast amounts of time into the intricacies of processing data and utilizing statistics, but paper scouting gets the job done and gets it done well.

Re: Best Ways to Scout
 

I have two answers with the first reflecting the goals of FIRST, that is to really test your abilities and to learn new things. I encourage those building apps, networking, automating processes, doing statistical calculations, etc.

And second, ideally for my team, I feel like three things needed to happen within scouting which is to utilize the FIRST FMS stats fully (develop automated processes and visualizations to pick out not only the best teams, but overlooked teams for potential 2nd picks), develop qualitative intelligence of the teams (through pit scouting, watching matches, drive team interactions, etc), and form bonds with the other teams.

You can do some pretty amazing things with the FIRST ranking data, a copy of Tableau, and some Excel magic (and also R). Our quantitative scouting ended up being scraping ranking data (which includes auto, defenses, goals, and scale points), calculating an OPR-like value for each of those, and visualizing in Tableau. It is very simple to filter and sort in Tableau if you like a more exploratory way to identify teams, or you can pre-build your analysis.

We looked at the data Friday night (after 8 matches) and Saturday (after 11). In retrospect, I would have liked to share information earlier between the quantitative and qualitative scouts. We did this at the end of the day but could have done it another time and had an extra iteration before the scouting meeting. By sharing information, I mean that from the data you can start to see which teams would be best to watch more closely (particularly for things that aren't captured well in the data -- like breakdowns, driver skill, etc), and that from observations you may find some teams that don't pass the eye test that isn't captured in the data. For example, I watched a robot that rated 14th by OPR-like stats in one of their final matches seem slow and indecisive.

Every team is capable of scouting in this way. Certainly, teams with more data have an advantage, although I think slight as sometime it doesn't make the picture clearer than aggregate data and the eye test. Next step to use the free data fully is to pull all the match data and analyze it. A team's ability to capture the tower was an additional thing we looked for in the best teams (with low goals being important, but not counting as much in goal score).

I regret this year not having as many students working with the data for scouting, given the educational opportunity. We did switch systems following our first regional where we were paper-based. I'd like all the scouting team students next year to know how to extract the data from the website, be able to do basic tasks in Excel, be able to build their own dashboards/plots in Tableau, and more but it is a start.