How does your team scout?

[maverickfan138]

Moderators, if this topic is in the wrong forum, please move it.

The title is pretty straightforward. How does your team scout at competitions? Does your team have any special methods or equipment(like computer software) to aid in scouting? What kind of info does your team collect? Also, how is your info used to make your pick list?

This year, 573 was basically a mess. We scouted differently at every competition we attended. We always match scouted and pit scouted, but the ways we match scouted changed. In Ann Arbor, all we had were match sheets with criteria like points scored, which zones the bot played in, if they hung and were successful, extra comments, etc. This was decent, but we didn't feel that it was sufficient. At Troy, we added 'report cards'. Basically, we had similar criteria on the report cards(scoring, hanging, bump, tunnel, etc) to what was on the match scouting sheets, but instead of numbers and yes/no's, we marked down letter grades. The idea behind them seemed great. At MSC, we used the report cards more heavily, but soon found out that the report card system was flawed. There were teams who had D's and F's on the report cards, yet were placed high in rankings. In Atlanta, we decided to do away with the match scouting sheets and just take straight notes on robots. We filled out the report cards, but took them with a grain of salt. We were then reprimanded for our method and switched back to the match scouting sheets, but still took straight notes instead of numbers. Down in Atlanta, we discovered that the report cards were even more flawed that we previously thought.

I cannot speak on how we did our pick list in Ann Arbor and Troy, but at MSC we basically sorted our report cards into a pile with our first pick on top and our last pick on the bottom. In Atlanta, we made sure our photographer took photos of every robot and uploaded them to a computer. We started with a do not pick list by going through all 85 teams in Curie and determining by their robot whether we should consider them or not. We then started at the top of the list with the top 20 or so teams that stood out in our memory and in our notes. We then sorted a few more teams before coming up with our final pick list. We only had around a 30 team pick list and the rest were just stuck in there because our coach wanted an 85 team pick list. This last way seemed like it was the more effective and thorough, even though it was incomplete and it took a long time to make.

So there's 573's story. What about the rest of the FIRST world?

[apalrd]

IKE posted parts of our scouting system in the CD-media section. At one event this year he and his scouts were 23-24 correct picks (not in order), and the team that wasn't on his list was out in the first round.

For match scouting:
We scouted by "good things". These include passes and scores, equally. This values a mid or far-zone bot as much as a front-zone striker, so the ones who steal the balls and pass get as much credit as those who finish the job. We had a sheet for each team, and scouted six matches per team. One sheet was dedicated to them, with six squares for data. On the back, the scouts would write comments on their performance and opinions of the teams. We also noted if they hung.

For pit scouting:
I don't know as much about this. We had a pit scout who took notes in a notebook.

For pick list time:
On Friday night, six mentors would go to dinner with the scouting data and make the pick list. In Atlanta, we had the discussion in the hotel and involved more mentors and students, talking about each team that was decent, picking our favorite first and second picks, etc.

EDIT:
Here is the link to the CD-media file. http://www.chiefdelphi.com/media/papers/2370

[XaulZan11]

I think you can get some good information if you use the search function. But, here is how 1732 scouts:

We (usually myself and around 3-4 students) start our scouting discussions about midway through build-season. The first step is to decide what to scout. In my opinion, this step appears to be skipped/skimmed over the most by teams. We approach this question from two angles: If I'm developing a match strategy, what do I want to know about our partners and opponents, and If we are doing a picking list, what information will we need to pick good partners. We list all the possible ideas and then debate if each one is actually important or not. This year we decided we needed to know: starting zone, goals in auto, balls advanced in auto, goals from each zone (close, mid, far), balls advanced (far-mid, far-close, mid-close), successful hang, time started hang attempt, where they hung, and number of times they changed zones (this was later taken out after we never used it at Wisconsin. It was replaced with how they changed zones). We also wanted a spot for 'general comments' (such as defense).

The next step is how to record and keep track of all the data mentioned above. Like last year, we decided to do a notecard, folder and excel system. Each match, a scout fills out a notecard for one team with the information above. After the match, some of the data (starting zone, auto goals, auto advanced, total advanced, total goals and successful hang) is recorded into an excel file so we can see how their averages stack up and easily see if they numbers are improving or getting worse. Then, the cards are placed into an accordion folder--one slot for each team.

At the competition, we have 6 scouts at a time fill out the notecards/excel file. We also 'pit scout' by taking a picture of every team and also recording their drivetrain (may sound a little pointless, knowing their drivetrain contributed to the selection of 171 and 3352 this year). At lunch breaks and on Thursday and Friday night, myself and a couple of the scouts create the qualification match strategy, using the notecards. Depending on how teams do their previous matches, these may be slightly changed. One of the scouts then informs the driveteam of the strategy before the match.

The formation of the picking lists actually starts well before the event. Myself and the main scouts/strategists have significant discussions throughout buildseason, especially after the robot is shipped when we know our strengths and have some idea of how the game is played (by watching scrimmages and some of the early regionals). Having those thoughts in our mind, all the scouts and drivers meet on Friday night to make the picking list. We go though each team, show a picture of their robot, run through their statistics and read off the comments. Each team gets placed in one of three categories: Yes, Maybe, No (names don't really matter, can do 1,2,3). We then go through each category and further divide those teams into groups, until we have all the 'yes' and 'maybe' ranked. I like to have around 35 teams on the picking list on Friday night. On Saturday morning, we only scout those 35 teams and note if teams are doing better or worse than their rank. We then make the necessary changes to the picking list at the end of matches. Note: we make a picking list regardless of our rank. In the past we have been picked by a team that did not have a long enough picking list for the 2nd pick, so our list was used. Additionally, it is good to method to review each team's strengths and weaknesses for match strategy.

During the elimination matches (if we are lucky enough to be playing), myself and one or two of the other main scouts/strategists try to sit near the field with our alliance's scouts/strategists to start developing strategies for the next matches and rounds.


Note: In 2009, we did not have 6 scouts, but we used the same method. Although the statisics were certainly not as accurate (because we scouted about half of every team's matches), I think it still worked well enough.

[J93Wagner]

Team 93, at least this year, reused a CakePHP database as a starting point, then went on to put in a really slick idea, using a fast-read scanner to enter in the data. In short, a scantron sheet. Three minutes of basic instruction of what to look for and how to fill in the bubbles (although I don't really think they could be called bubbles since they were squares... ) and they were nearly ready to take over for someone. The only downside I notices, it took awhile to fill in those pesky bubbles (squares). At any rate, with good reliable data, it wouldn't take long to figure out who the outstanding robots on the field were.

So if you were taking a look at the sheet you might see:

Scores

Near: [] [] [] [] [] [] [] []
Middle: [] [] [] [] [] [] [] []
Far: [] [] [] [] [] [] [] []

Then all a person would have to do would be fill in the number of []'s that that robot did by each zone. We used the same method with transfers and other types like autonomous mode as well.

Because some traits, like defence, can't be qualitative (or is it quantitative?), a comments box for whatever you want to say is included as well.

That's the major stuff, although the []'s for the program to run properly were included. A.K.A. Team number, match number and type of match.

[penguinfrk]

At the beginning of build season (part of our build process) we list all of the possible objectives to the game. This year, it was scoring goals, elevation, and then suspension. Along with that, we also listed any "obstacles" or extra functions: autonomous, climbing bump, under tunnel, defense.

Our team, at least so far in its history, only attends the Microsoft Seattle Regional, a week 4 or 5 event, so all of our scouts are expected to view webcasts or videos of previous events. We make a finalized scouting sheet, adding whatever we didn't initially list as an objective/function, along with driver skill, human player skill, and W-L-T record.

During the competition, Day 1 we do pit scouting as a priority. Clipboards with the pit map on the back and with pens tied to them (nicely done by one of our mentors) are given to about 6-10 scouts, depending on whoever isn't in the pits helping to pass inspection. Of course, we have a relatively large team (upwards of ~35 students at competition each day and total membership in the 60s), so we have plenty of scouts.

Day 2, the real matches are scouted with one person on each robot. 6 scouts are watching each match, a scout team leader should (I say this because the effectiveness will ultimately depend on the responsibility of this leader, as we learned the hard way this year) be sorting teams. This scout leader should be the most knowledgeable of all the scouts, and if not sorting the actual scouting sheets, should be watching every robot and sorting them into Whitelist (Yes!) or Blacklist (Absolutely not!). Other scouts are having some sort of downtime - alternating scouts each match.

Our mentor had connected to the internet there, so we had full access to the match results from the stands (no need to look at the scrolling standings). If no internet access is available at the competition venue, a hard copy would be printed (we printed it anyways) from day to day. From there, highlighters are your friends!

After day 2, the photographer is expected to have collected all of the robots on to a single storage device (obviously compatible with any laptop we bring). The scout lead should compile a list of 24 (number of teams in the eliminations, to ensure not all will be selected) robots most compatible with yours. We filed them into top tier (expected alliance captains, first picks) and second tier (expected 2nd picks).

On day 3, our team (mostly PR) talks to top tier teams to sell our team. The scouts in the stands scout only our whitelist (top/second tier) robots. We have one person in the pits at all times to talk to teams that swing by. If we're picked 1st or end up alliance captain, we use our second tier list to pick the next round.

[MasterRobot]

For our team, team 2438, we have tried and implemented many systems for scouting over the last few years.

A year or two ago, we started off with a beautiful access database. On the first day, some people would go around and take pictures of each robot to add to the database, then robots were given values based on their performance in categories such as driver skill, offense, or defense. This did not work. I am not how teams use the numbers if they do it this way, but for us, we found the number system impossible to read and understand. Someone could if they wanted too, but the team didn't want to do that, drivers wanted qualitative things, and the captains only wanted a list.

This year we focused on those two goals in creating a scouting database. We use the first day, and part of the second day to take qualitative data, and strategy data on all the teams. We used an html based database with a person who ran from the pit to the arena updating the pit scouting computer with data from the matches. For ranking, we ranked every robot on a 1-6 scale. 1 being the best robot in the match, 6 being the worst. That with the data on the zone taken into consideration, yielded a list of the best performing robots per zone.

We so far like this system because it is simple and friendly to everyone. Plus, this can be used with anywhere from 2 to 10+ scouts working or ranking. The drivers get the strategy and qualitative stuff they want (no confusing "5" for speed numbers) plus we get a list of the top 24 robots for picking. For us, scouting needs to be simple to work and any scouting system, computerized or on paper should be tailored to what the team needs and wants in order to succeed.

[jblay]

I find that, especially for a team that has not done in depth scouting before, a computer system or anything beyond the simplest thing possible will cause chaos. In my opinion the thing to do and the thing that we do is keep the data as simple as possible. Just cover the basics for each match(balls scored, hang, can they go over the bump or under the tunnel, etc...) and then leave tons of space for your scouters to comment because that on its own will allow you to pick up on sleepers and understand the capabilities of each robot.

Things like charts and graphs using computers aren't really that useful. All they do is save you time but they do not help with making the right choice for selections and the comments are key for developing good match by match strategy. All creating complicated computer programs and scouting sheets does is risk chaos in the stands and the benefit is minimal. When you have the simple raw data you can figure out which team is better fairly easily and graphs aren't really necessary when you can just look at the numbers and compare two teams.

The comments are the key to developing good strategy and if you try and make everything complex you sacrifice this and end up with this raw data that you would have regardless just less efficient but the difference between the teams is so great that this efficiency is never necessary.

We used to always try to have a complicated system but then I came along and made everything as simple as possible and I think we have one of the best strategy approaches from the teams that I have interacted with throughout my years in FIRST. In my opinion too many teams try complicated systems and only the best teams pull it off because they have the man power and the member dedication. I think that for most teams out there something simple is the best option.

[Chris is me]

First, we figure out what an alliance needs to win the match. We try to be as specific as you need, but no more. Being too specific constrains you too much (committing to "two scorers and a defender" in Lunacy was the downfall of many teams, for example), but being too general doesn't form the alliance you want. In Breakaway, we wanted an alliance that could shoot 6 in auto, had an agile striker, high traction feeder with excellent ball control, and a defender that could at the very least clear balls in auto, with a hanger or two thrown in there if possible.

Then you figure out what stats will best demonstrate who is the best at the roles you've dictated for your alliance. In Lunacy, +/- was a good indicator for almost any alliance. Breakaway, it got more complicated. Auto kicks, front scores, mid feeds, pushing, hanging, etc.

Then you assemble your list or lists based on your data and who's best at what you want. You figure out what you want in a first pick and what you want in a second pick, then you make them. This year there were two general approaches to the two picks, mainly the "front last" second pick and the "back last" second pick (even if they started in the middle to allow the primary scorer access to 3 balls in auto). Alliances like 254 / 233 / 3357 picked their frontbot last, while alliances like 2056 / 1625 / 3138 picked their backbot last. This depended on what was available in the field, what choices people had, etc. I know both times for my team we ended up picking the backbot last, but for different reasons. 1124 was inexplicably still around for the 12th pick of the draft (granted, this was before they changed their code and became the frontbot they played as at CT and champs), and they had an auto that could theoretically clear 2 balls out of the back. At CT, the much deeper field left 173, 3125, and 102 available for that 12th pick; 173 had a lot more consistency on Saturday morning than 3125, so the choice was made to pick them.

[Bethie42]

The major factor for our team has always been number of scouts....in past years we've had a small team....small as in, 5 or 6 students total. My first preference would be that you have a fairly large scouting team, enough people in the stands to watch EVERY robot in EVERY match.

My first two years as scouting captain I was also the ONLY scout, those were fun times....

This year [Autodesk Oregon Regional] was the first year I had a laptop. Don't underestimate the power of one really dedicated person with a laptop. I was also very fortunate this year to actually have a second person in the stands scouting with me. By the end of the competition [heheh] we had the system down to a science: my scouting partner would watch the robots on one alliance [easier to do that, than by zone, because of color, and the fact that robots don't always stay in their assigned zones], and I would watch the bots on the other alliance. I had the laptop, and would enter data on my alliance during the match. Scouting partner would feed me data after the match.

The only way this method would work [with no special program, just an OpenOffice spreadsheet] is if you have The Scout who does nothing all competition except watch every single match. The Autodesk regional is one of the biggest I think [60+ teams this year] but I still had a good handle on knowing each and every team and robot on the field.

Over the years I've become less interested in pit scouting, at least at Regionals. Anybody can say they can score in autonomous, but only the matches will show whether they DO. The chief value I see in pit scouting is establishing a rapport with other teams, and it can be hard to judge early on who will be in the top seed. Of course, by the end of the practice matches at AOR I had 'the cozying-up list' which included the teams that went on to place first and second.

I would encourage teams to not just shuffle off scouting on the members of the team who don't have anything else to do: it really takes a lot of dedication. One person has to be willing to spend the entire competition [before breakfast, during matches, at lunch, after the matches end, at dinner, after dinner, at all hours of the night] doing nothing but become familiar with every robot on the field. It's fun.....

[miniman]

Team 701 took a leaf out of team 100's book. We decided this year to base out scouting around the Nintendo DS system. We did this for a few reasons, the first being they are light weight and portable. They are readily available, day before we leave for competition we tell every one to bring their DS and charging cable (we try to end up with 7). Lastly the data is already digital data that doesn't need to be copied over from any kind of hand written sheet, just download and analyze.

In choosing what areas to scout we focused on purely empirical data. Stuff that isn't based on opinion. Things like scores, number of balls advanced, start area. No robot rating by the scouts, no opinions, just cold data.

The process we used was very simple, during prep/practice day pictures were taken of all the robots and the match list was digitized. Pictures and list were then saved to 6 individual SD cards along with the software written during build season. the SD cards were then inserted in the NDS (via special adapter) so that the software could run, and access the pictures and list. During a qualification match 6 scouts, each with a DS would each scout the specific robot that the DS was assigned (based on the match list). once all the data was collected and saved to the SD card it was transfered to a laptop, run through Microsoft access, analyzed, and manipulated to give rankings based on what our alliance requirements were.

Next year we are looking to overhaul our software, moving from C to C++. We are also hoping to take advantage of the wireless capabilities of the DS.

[Wildcat]

what i did:
pre-season: thought up systems and edited previous sheets as needed for any off season competitions. made up a scouting systm to integrate everyone into it with an equal amount of work, then used it @ kettering kick-off
build season: since there was so little to do i focused heavily on scouting. i spent hours on hours making sheets and presentations for my team on how the sheets should be filled out.
competition season: with the acception of kettering scouting was good on my team. i paired up vets and rookies so the rookies could learn how to do eveything. that worked for 2 out of 3 competitions. at the third one we had too few people to pair up so pairs were only done for pit scouting while match scouting was solo. Data gathering wise it was straight up pen and paper for the most part and at two of our competitions we had a video scouting system where we would record all of the matches for that competition to help fill in gaps where data was missing, in addition to that our videos made it to the Blue Alliance. An additional thing that was done by my scouting team was that we went out to the troy competition where we were not competing just for the purpose of scouting for states. Also a side note: if your scouting department wants to get on good terms with the rest of the team organize a pizza lunch @ competition, it was done on my team and was a big hit/
Off-seasn: i plan on devloping beta systems to test during off season and further develope for the next year.
If you specificly or any other team needs help with scouting feel free to approach the 862 scouting team at competition, we are usually sitting in the very front of the teams section or at the very top.

[xxjgdancexx]

The Technomancers have a scouting team to scout both the pits and the matches.

Pit Scouting-
The scouting team is divided into pairs and each pair has a folder with ten or so teams scout. The number, of course, depends on the number of teams at the event and the number of people on the scouting team. We find our assigned teams and ask them questions such as (and these are based on the Breakaway game):
-how long they have been a team
-if their bot could play front, back, centre, or more than one
-how they scored balls and any other offensive moves
-defensive abilities/moves
-how many bumps they could shoot over (if any)
-if they went over the bumps, through the tunnels, or both
-type of wheels
-could they hang
-if they had an autonomous
-if they had pnuematics (and if so, where was it located on the robot)
-of their robot had any other outstanding abilities that they wished to share
-etc

For the matches:
Our scouting team would work in shifts (3 people would watch about 8 matches and then it would go to the next shift). Each person was assigned a section of the field (front, back, middle) and would take notes on the two robots in their section. Such notes would be:
-if they bot had an autonomous (and if they scored)
-special moves or abilities
-good traction
-if they helped other bots or were assisted
-how many they scored and how
-defense and offense (and how well they could do in these areas)
-which alliance won the match
-score
-etc

For both the pit and match scouting, we had spreadsheets that our scout master typed up for us to use.

Also, whenever we have finished scouting all of the teams and the matches, then we discuss as a group, what teams had the best robot (based on scoring, overall reliability, extra abilities, defense, offense, etc) and our drive team would also give us input of the teams that were easiest to work with at the driver station as well as the speed of the human players. We separated the team sheets into piles of "definatley like" (which was more like our "to make us feel better" pile since most of them were already ranked in the top 8), a "like," "last resort," and "definatley NO." This really helped us to make our list for alliance teams.

It does take a lot of dedication as some others previously stated. It is basically all that the scouting team and I did at teh competitons but I loved every minute of it. You get to meet awesome people, watch matches, collect buttons, etc

[rotolomi]

Team 1676 comes from a district that provides its students with laptops, so we use a computer database to store our scouting data. At the arena we connect to the database using Ethernet cables (since wifi was banned we had to blast to the past) and fill out a scouting form that includes team number, match number, alliance color, abilities, goals scored, etc. Student are assigned one robot position and alliance color (for example: red defense) and they scout those robots for a few matches before switching off. We also offer all the match information to all other teams at a regional and other teams in our division at the World Championships.
As for pit scouting, we send a few students with their laptops to other teams pits and record robot specs on an Excel document- pretty basic.

[Vermeulen]

This year, we had some students build and code an optical scanner system, which was built in the week before our regional out of some scrap angle iron, wood, and a digital camera. By Champs, we had (almost) fine tuned the software, and made a scantron-like scouting form, and 80% of our picks were in the Curie finals. We might release the software at some point, too, after we get rid of some bugs.

[TEE]

I asked someone on team 67 how they scouted at the Michigan State Competition, and their scouting was the best I've seen, and the easiest to implement; try hunting one of them down sometime and asking about it (I'll probably explain more later)