TOO much scouting info???

Hi,
Is it possible to collect TOO much information about robots and their teams?

Scouting

Consider all of the variables associated with six robots, six drivers, six assistant drivers, six coaches and six human players. The interactions during dozens and dozens of pre-qualifying matches… Luck, bad luck, good ref calls, bad ref calls…you end up with literally thousands of pieces of information.

Analysis

Someone, or several people, or a computer program using some algorithm, with a little bit of ‘fuzzy logic’, must crunch all of this data to yield important information about game play and team selection for the finals.

Strategy

Someone, or several people, decide how you will play the next match. This often involves meeting with your two alliances teams to decide who will do what (if everyone is cooperative). Perhaps your conclusion is different then the conclusion of one of your alliance partners.

Whew!!

Assessing Results

Do you go back after a competition, sift through all of that data, your concluding strategies and determine what information was really relevant? Was it really necessary to collect all of that information? I know it looks impressive but was it really important?

Conclusion

We walked down this path this year at two regionals. We have narrowed our scouting information to just two averaged variables that our drive team captain needs to know about our alliance partners and opponents to direct our team during those two-minutes of game play.

Have you applied any ‘data reduction’ processes after a regional to determine what was truly important?

Regards,
ChuckB

Is it possible to collect too much information? Yes, but if you have an effective scouting strategy, you will identify the extraneous information during/after your regional competition and eliminate it. The scouting process should adapt as you proceed through your competitions. It should become more streamlined to record and focused on those characteristics that are important.

You may have identified two points that are important to your competition team, but there are many more points to consider if you are one of the alliance captains picking partners. “How our robot plays with/against yours” is not the same as “Which of these equally capable bots would best fit in our alliance?” That’s where some of the finer details of scouting come into play. The top few bots in the division will be easy to pick out – how do you plan to distinguish between the 18 to 20 next most capable bots when you work off two datapoints?

I always find it amazing that teams send their alliance captains down to the field unprepared. If the 8th Alliance captain doesn’t have 24 names on their list (or 23 if they don’t include themselves), then their scout/strategy teams have let them down. Yet every competition you see captains looking up into the stands trying to guess what their team is telling them. This means (since the top seeds can pick each other) that teams seeded down through 15th, could be picking, and should be prepared to do so with a list of 24 names.

Folks, I don’t care if you placed in the top 8 after qualifying rounds, or if you were bumped there because of other alliance picks, YOUR TEAM BELONGS THERE BECAUSE OF THE GREAT AND CAPABLE GAMEPLAY YOUR ROBOT AND COMPETITION TEAM DISPLAYED. Don’t arrive unprepared to put together a division winning alliance, but be thoroughly prepared to put together the best one you can to get every spectator in the audience on the edges of their seats.

Win, Lose or Tie, but do it in a way that FIRSTers will be talking about for years to come!

I don’t think it’s possible, personally. You can never know too much information about a robot. It’s a precaution to know a lot because in the long run it’s good to know how the team’s robot works exactly, and keeping careful watch on all the robot does is essential.

No worries, we won’t consider you psychos…or…:ahh:

Our “tried and true” ultrascientifictechnology:

People watch matches and make notes on which teams impress.

Sure, seeds and wins and all that can be used as a starting point, but let’s face it: the numbers can mask what is really important.

  • Which drivers adapt to changing fortunes and tactics and keep their focus?
  • Which robots are productive, whether or not their alliance partners are?
  • How many tetras can a team score in a match when their tower isn’t working right? (hint: one team (cough56cough) did 4 in one round in Philly with only part of theirs working)

I guess I’m just not that sharp after all - don’t know how to build a sorting algorithm to find that info in a table of Regional statistics. Maybe some of your more agile young minds can do it - if so, please feel free to make me feel dumber!

All the scouting data you could get isn’t worth beans if your coach or drivers don’t use it. They are your customer. Find out what they want to know and give it to them. For us, I mainly want caps per match. As the coach I try to get a look at my opponents personally - especially the high risk ones. Find out there weakness/strength. My scouts will fill me in also, but usually our communication is limited by distance and time factors. At Nationals it’s going to be very difficult to get the info from them while they’re in the stands. We might try a 2 fold effort. I’ll watch teams off my list, they’ll watch up there in the stands. Then I’ll try to get the caps per match from them after each of our matches for our upcoming ones.
If we decide a defensive effort I need to know how the opponent does his thing, so I can get it stopped properly. But having tons of data on every team is overkill. Proper assessment on the specifics is all that’s needed - according to your drivers and coach.

There is no such thing as too much scouting info as I have learned this year. There is, however, such a thing as extraneous or unimportant scouting information. You can determine that probably by a simple test: can you get the exact same information watching the robot in action from the stands AND talking with team members in the pits? Also make sure scouting info relates to the game and not to tech specs of the robot.

-Joe

Good point! A classic rookie mistake is compiling tons of technical info and team publicity, in the belief that it will tell them how the team will perform on the field. I believe that scouting is best done two ways: watching matches and talking to the drive teams/student team leaders.

As being one of two people in charge of strategy for Heatwave I would have to say that no, you can never collect too much info. I do admit that there are only two or three variables we look at for every team for every match, but then we may look at other more specific variables for given teams in given matches. The way I look at it is that there may be something we look for in every robot in every match and we may only call on that information once and it could make the difference, or not, but its worth having. Being the one to “compile” all the information and then relay to our drivers my best prediction of what our opponents will do and our alliance partners can do, I look at nearly all the data we have then cut it all down to what is relevant.
When it comes down to it I prefer strategy to only be limited by personnel, time, resources, and will power.

When you ask if it is possible to have too much scouting info, I must say yes and no. The reason for this is because it is very possible to have too much information that is not necessary or not helpful, but there is no such thing as too much quality information. Example: The color or armor of someone’s robot is not all that important to me when I am looking at forms for an upcoming match, especially since the game this year does not necessarily require heavy armor. Important, quality information would be how many tetras a robot can stack per game or how reliable it is. It is better to have a smaller amount of good information that an excess of useless information. I know I modified our scouting sheets for Atlanta to get rid of some spaces we had on our scouting sheet that turned out to be insignificant when planning strategy for a match. This allows scouts to concentrate on the more important information. So, yes it is possible to have too much info if it is bad or useless. It is not possible if the information is important and relevant.

As head student scout for my team, I think that there is no unimportant data except maybe match scores.

We do things in an “original” way: Thusday is spent talking to teams and taking pictures. Friday, finish up Thursday’s job and take notes on the matches. 6 people are each assigned a driver station and watch only the teams in that driver station, including us. Someone usually takes video of how we do to watch later and look for mistakes that we make. Friday night we meet, watch the video, evaluate the data for the day, and decide what we would like Saturday’s strategy to be based on the data. We also get a list together of who we will probably want to pick if we get that far. Saturday we watch the rest of the matches, scout them, and revise the list accordingly.

The most important information is:

  1. Match number- helps determine if a team has improved
  2. number of tetras capped by that team
  3. number of tetras brought onto the field by that team
  4. penalties if seen or announced
  5. auto mode
    This info is not necessarily listed in order.

See what I mean about not having too much useful data, ever?

Excellent feedback from everyone!! Reducing all of that scouting info to a clear statement of how to play that next match is pretty challenging. An then the drive team must be willing to accept it as Swampdude pointed out.

[my sea story]
At Palmetto, by the end of Friday competitions, we were doing poorly in the qualifying rounds even though we had these things going for us:

1). We spent about 6 hours the day before making 3 major mods on our bot that we learned from in Florida. This gave us a great bot.
2). We watched and filmed every match and recorded lots of scouting info similar to that suggested by previous posts.

We posed this question to our stategy and drive team that Friday night? Why were the top 5 robots ranked in the top 5 after only 6 or 7 matches? Our robot was now as good as these robots, but what was the difference?

The team (students) went to the hotel that night, watched several matches with the top 5 bots, watched our matches, bounced the results with all of our scouting data and came up with this significant strategy change.

  1. We were to play in a specific (?) manner.
  2. We were to reduce all scouting data to just two (?) variables.

The change in our game play was incredible. Finally, after two years, we were doing things right.

NOTE: We’ll share the (?) as soon as Radioshack unlocks their vexrobotics.com website. :wink: (something like 8 d, 7 h, 34m, 29 s)

So, reducing collected data (tons of it) into something applicable to a 2-minute match seems to be the key element we were missing. We look forward to seeing a lot of you guys in Atlanta and finding out if our team’s efforts will pay off. To the rest, we’ll see you in 2006.

Good luck,
ChuckB

I noticed you guys improved tremendously on Saturday. In the finals you guys were rocking. I thought we were going to be seeing you in the finals. I’m surprised some info is what you needed. It seemed you were more limited by design. But you looked like a totaly different team on Saturday.

Are you still sharing?

Last year at Vegas, we got 4th seed as rookies with NO scouting data and NO idea about who to pick. We made it to semis, but could have gone farther, with better info!

In fact, I’d like to see posts or links of ALL teams scouting plans and tools.

2 best pieces of advice I’ve seen so far are:

  1. Collect PERFORMANCE data, because that’s all that matters, in the end (Ask the drive team what info they think is most important on the field)
  2. Assign (if possible) a scout per team in the stands to get match data that can show improvement or decline over time

As the Scouting Leader of our team for the 2005-2006 year, I cannot put more stress than anything on two factors: Communication between the drive team and the scouts, and the quality of people scouting.

One of my goals this year was to analyze our scouting system last year, and over the past years to look for potential improvements and put them in place. Here are a few of the things I’ve noticed, in support of statements above.

  1. No matter how much info you can collect, your drive team doesn’t need it all. They are your drive team because they have proven that they can think quickly and make informed and fruitful decisions for the ultimate purpose of success. Our job as scouters is to give them what they want and when they want it, and not hinder them with unnecessary information. Thus, I am of the belief that you can definitly have a large amount of extra information.

  2. Your scouters must be mature and committed individuals. Scouting is not just a “thursday-friday-saturday” commitment with fun and games in between. If you want your team to be prepared and efficient during competition, you as a scout must put in effort. How?

  • Attend strategy sessions. One of the things you find out from teams is their strategical priorities and how they mesh with your own. To do this, you must understand your team’s strategy.
  • Do as much prescouting as possible. A reasonably accurate list of teams is available online months before competition, and small changes can be made on-site as needed. So, be flexible too!
  • Brief the drive team / tech team at least weekly on your findings. Tell them about your plans, your organization, and stay updated on their strategic and structural priorities throughout the season.
  • Everybody loves socializing. It’s great, I like it too. But your team must come first if you are a scout. You can socialize in the evening at the hotel, or when the day’s scouting work is finished (if it is) or you can meet people quickly when you scout their teams.
  • Last but not least, although every single scout I know is very able-minded and capable, it is very advisable to have a mentor specifically tied to your scouting sub-team. Mentors can offer insight and guidance, as well as mediate any conflicts that arise. They shouldn’t do jobs for you, but they can help you stay on top of your work and keep you from being overly stressed.

Above all, good luck! Competition is awesome. Our scouting system last year worked very well, and I would be happy to discuss with anybody interested the specifics of having a successful scouting team.

I don’t know if “too much” is the correct term for the extraneous information gathered during scouting. Any information is “too much” if you don’t use it. While I agree that some information is usually very irrelevant (like knowing what drive system the robot has, etc). Sometimes it is nice to have that information to reference even though it is never actually used. It can be especially helpful when meeting with the drive team to discuss what has been observed all day. This information can help drivers visualize alliance partners without having actually seen them in action (because the drive team is stuck in the pit the whole time, so they never actually get to see robots competing).

We have two main components to our scouting system: pit scouting and match scouting.

Last year I sat though every single seeding round and watched every robot’s performance. I developed a system in the stands to watch the teams. I ended up knowing how well every team was able to perform in the heat of competition. This was useful that night when we had our driver/scout meeting to discuss potential alliance partners. We recorded the alliances and the final scores, as well as any match notes. This helped us find a trend in how well certain robots performed with certain partners.

In the pits we have a group that photographs every robot and takes technical data. This information is mostly used to keep track of “who” teams are as well as describe the other bots to the drive team like I mentioned earlier. Pit scouting is also important for introducing our team to other teams and putting ourselves in their mind. This is useful later when they are considering alliance partners, etc. It is also just good practice and a good display of sportsmanship.

In the end it all comes down to how you use the information you acquire and how well known you are around the competition. If no one knows who you are then you have little hope of being chosen first (reminiscent of days on the playground). Perhaps the most important thing to do while scouting is getting to know your competition, because they could end up being your partners.

If you do plan to collect a lot of information during matches, try to develop a checklist for your scouts that will collect the information easily. Then have training sessions before competition. If possible, use some of the streamed regionals if you aren’t competing the first weekend.

There is nothing worse than the scouts writing out notes longhand, and having them miss half a match while doing so. (What happened? I was writing stuff down!)

It’s interesting that you mention scout training. Our team competes in another robotics competion in the fall and I actually used that competition as a pseudo-introduction to scouting. I didn’t get to train and organize as much as I would have liked because I had other responsibilities, but at least it gave the newbies a taste of scouting.

It’s all good and well to use points scored as a criterea, but don’t do what a lot of teams do and overlook defense when you’re planning.

From what I saw, (I didn’t scout) our team’s scouting was messy and slow. The papers ended up in a big messy pile and was hard to read the writing at times, also the team members had trouble seeing the field. This year, binoculars and (maybe) a database (if I get around to learning acess) run on a 802.11b network with better dbi directional antennae for pit <–> stands communication. But, then we need the school to lend us ten(ish) laptops. Ideally I’d get a decent camera and film all the matches and connect it to a high capacity server/media center computer, before you mention the instant replay thread, remember, we aren’t making juding calls on who’s zip ties are in the triangle here, just deciding strategy, which (I think) doesn’t need as good resolution.

True, it’s the complete picture you’re looking for. I know we’ve run the gambit as far as good ways to characterize a team and bad ways to characterize a team - one thing I especially note is that team members like to tell you they can give you the world (if picked) and end up doing nothing on the field. Noting what they say they can do and what they actually do (and how they do it) is critical, but as many people have said already it comes down to what info your drivers have when it comes time to pick. Ebolagirl had a good point - drivers have usually seen nothing, and all your scouting data is useless without some kind of way to say, “Pick this team if they’re available.” Having tons of raw data from the scouts is kinda useless without a rubric for scoring that data, and then it turns into a matter of how well the rubric was designed. In the end it’s all about gut feeling anyway; at least at Lone Star when it comes time to pick the third alliance member there are a large pool of qualified teams who all rank very similar - how do you know who to pick then? :ahh:

-Danny

Why tape all the matches? Way too messy, unless you have a lot of tapes, and generally lacking quality to see what’s going on with everyone.

The MOST IMPORTANT thing you can have is a picture of a robot. Really. With a few simple stats you’ll be golden becuase pictures say a thousand words. Also usually there’ll be one person in your scouting crowd who knows every team and their robot, or collectively you’ll be able to do that.