TOO much scouting info???
 

Hi,
Is it possible to collect TOO much information about robots and their teams?

Scouting
=======
Consider all of the variables associated with six robots, six drivers, six assistant drivers, six coaches and six human players. The interactions during dozens and dozens of pre-qualifying matches.... Luck, bad luck, good ref calls, bad ref calls...you end up with literally thousands of pieces of information.

Analysis
======
Someone, or several people, or a computer program using some algorithm, with a little bit of 'fuzzy logic', must crunch all of this data to yield important information about game play and team selection for the finals.

Strategy
======
Someone, or several people, decide how you will play the next match. This often involves meeting with your two alliances teams to decide who will do what (if everyone is cooperative). Perhaps your conclusion is different then the conclusion of one of your alliance partners.

Whew!!

Assessing Results
=============
Do you go back after a competition, sift through all of that data, your concluding strategies and determine what information was really relevant? Was it really necessary to collect all of that information? I know it looks impressive but was it really important?

Conclusion
========
We walked down this path this year at two regionals. We have narrowed our scouting information to just two averaged variables that our drive team captain needs to know about our alliance partners and opponents to direct our team during those two-minutes of game play.

Have you applied any 'data reduction' processes after a regional to determine what was truly important?

Regards,
ChuckB

Re: TOO much scouting info???
 

Is it possible to collect too much information? Yes, but if you have an effective scouting strategy, you will identify the extraneous information during/after your regional competition and eliminate it. The scouting process should adapt as you proceed through your competitions. It should become more streamlined to record and focused on those characteristics that are important.

You may have identified two points that are important to your competition team, but there are many more points to consider if you are one of the alliance captains picking partners. "How our robot plays with/against yours" is not the same as "Which of these equally capable bots would best fit in our alliance?" That's where some of the finer details of scouting come into play. The top few bots in the division will be easy to pick out -- how do you plan to distinguish between the 18 to 20 next most capable bots when you work off two datapoints?

I always find it amazing that teams send their alliance captains down to the field unprepared. If the 8th Alliance captain doesn't have 24 names on their list (or 23 if they don't include themselves), then their scout/strategy teams have let them down. Yet every competition you see captains looking up into the stands trying to guess what their team is telling them. This means (since the top seeds can pick each other) that teams seeded down through 15th, could be picking, and should be prepared to do so with a list of 24 names.

Folks, I don't care if you placed in the top 8 after qualifying rounds, or if you were bumped there because of other alliance picks, YOUR TEAM BELONGS THERE BECAUSE OF THE GREAT AND CAPABLE GAMEPLAY YOUR ROBOT AND COMPETITION TEAM DISPLAYED. Don't arrive unprepared to put together a division winning alliance, but be thoroughly prepared to put together the best one you can to get every spectator in the audience on the edges of their seats.

Win, Lose or Tie, but do it in a way that FIRSTers will be talking about for years to come!

Re: TOO much scouting info???
 

I don't think it's possible, personally. You can never know too much information about a robot. It's a precaution to know a lot because in the long run it's good to know how the team's robot works exactly, and keeping careful watch on all the robot does is essential.

No worries, we won't consider you psychos....or....:ahh:

Re: TOO much scouting info???
 

Our "tried and true" ultrascientifictechnology:

People watch matches and make notes on which teams impress.

Sure, seeds and wins and all that can be used as a starting point, but let's face it: the numbers can mask what is really important.
- Which drivers adapt to changing fortunes and tactics and keep their focus?
- Which robots are productive, whether or not their alliance partners are?
- How many tetras can a team score in a match when their tower isn't working right? (hint: one team (cough56cough) did 4 in one round in Philly with only part of theirs working)

I guess I'm just not that sharp after all - don't know how to build a sorting algorithm to find that info in a table of Regional statistics. Maybe some of your more agile young minds can do it - if so, please feel free to make me feel dumber!

Re: TOO much scouting info???
 

All the scouting data you could get isn't worth beans if your coach or drivers don't use it. They are your customer. Find out what they want to know and give it to them. For us, I mainly want caps per match. As the coach I try to get a look at my opponents personally - especially the high risk ones. Find out there weakness/strength. My scouts will fill me in also, but usually our communication is limited by distance and time factors. At Nationals it's going to be very difficult to get the info from them while they're in the stands. We might try a 2 fold effort. I'll watch teams off my list, they'll watch up there in the stands. Then I'll try to get the caps per match from them after each of our matches for our upcoming ones.
If we decide a defensive effort I need to know how the opponent does his thing, so I can get it stopped properly. But having tons of data on every team is overkill. Proper assessment on the specifics is all that's needed - according to your drivers and coach.

Re: TOO much scouting info???
 

There is no such thing as too much scouting info as I have learned this year. There is, however, such a thing as extraneous or unimportant scouting information. You can determine that probably by a simple test: can you get the exact same information watching the robot in action from the stands AND talking with team members in the pits? Also make sure scouting info relates to the game and not to tech specs of the robot.

-Joe

Re: TOO much scouting info???
 

Good point! A classic rookie mistake is compiling tons of technical info and team publicity, in the belief that it will tell them how the team will perform on the field. I believe that scouting is best done two ways: watching matches and talking to the drive teams/student team leaders.

Re: TOO much scouting info???
 

As being one of two people in charge of strategy for Heatwave I would have to say that no, you can never collect too much info. I do admit that there are only two or three variables we look at for every team for every match, but then we may look at other more specific variables for given teams in given matches. The way I look at it is that there may be something we look for in every robot in every match and we may only call on that information once and it could make the difference, or not, but its worth having. Being the one to "compile" all the information and then relay to our drivers my best prediction of what our opponents will do and our alliance partners can do, I look at nearly all the data we have then cut it all down to what is relevant.
When it comes down to it I prefer strategy to only be limited by personnel, time, resources, and will power.

Re: TOO much scouting info???
 

When you ask if it is possible to have too much scouting info, I must say yes and no. The reason for this is because it is very possible to have too much information that is not necessary or not helpful, but there is no such thing as too much quality information. Example: The color or armor of someone's robot is not all that important to me when I am looking at forms for an upcoming match, especially since the game this year does not necessarily require heavy armor. Important, quality information would be how many tetras a robot can stack per game or how reliable it is. It is better to have a smaller amount of good information that an excess of useless information. I know I modified our scouting sheets for Atlanta to get rid of some spaces we had on our scouting sheet that turned out to be insignificant when planning strategy for a match. This allows scouts to concentrate on the more important information. So, yes it is possible to have too much info if it is bad or useless. It is not possible if the information is important and relevant.

Re: TOO much scouting info???
 

As head student scout for my team, I think that there is no unimportant data except maybe match scores.

We do things in an "original" way: Thusday is spent talking to teams and taking pictures. Friday, finish up Thursday's job and take notes on the matches. 6 people are each assigned a driver station and watch only the teams in that driver station, including us. Someone usually takes video of how we do to watch later and look for mistakes that we make. Friday night we meet, watch the video, evaluate the data for the day, and decide what we would like Saturday's strategy to be based on the data. We also get a list together of who we will probably want to pick if we get that far. Saturday we watch the rest of the matches, scout them, and revise the list accordingly.

The most important information is:
1) Match number- helps determine if a team has improved
2) number of tetras capped by that team
3) number of tetras brought onto the field by that team
4) penalties if seen or announced
5) auto mode
This info is not necessarily listed in order.

See what I mean about not having too much useful data, ever?

Re: TOO much scouting info???
 

Excellent feedback from everyone!! Reducing all of that scouting info to a clear statement of how to play that next match is pretty challenging. An then the drive team must be willing to accept it as Swampdude pointed out.

[my sea story]
At Palmetto, by the end of Friday competitions, we were doing poorly in the qualifying rounds even though we had these things going for us:

1). We spent about 6 hours the day before making 3 major mods on our bot that we learned from in Florida. This gave us a great bot.
2). We watched and filmed every match and recorded lots of scouting info similar to that suggested by previous posts.

We posed this question to our stategy and drive team that Friday night? Why were the top 5 robots ranked in the top 5 after only 6 or 7 matches? Our robot was now as good as these robots, but what was the difference?

The team (students) went to the hotel that night, watched several matches with the top 5 bots, watched our matches, bounced the results with all of our scouting data and came up with this significant strategy change.
1. We were to play in a specific (?) manner.
2. We were to reduce all scouting data to just two (?) variables.

The change in our game play was incredible. Finally, after two years, we were doing things right.

NOTE: We'll share the (?) as soon as Radioshack unlocks their vexrobotics.com website. ;-) (something like 8 d, 7 h, 34m, 29 s)

So, reducing collected data (tons of it) into something applicable to a 2-minute match seems to be the key element we were missing. We look forward to seeing a lot of you guys in Atlanta and finding out if our team's efforts will pay off. To the rest, we'll see you in 2006.

Good luck,
ChuckB

Re: TOO much scouting info???
 

I noticed you guys improved tremendously on Saturday. In the finals you guys were rocking. I thought we were going to be seeing you in the finals. I'm surprised some info is what you needed. It seemed you were more limited by design. But you looked like a totaly different team on Saturday.

Re: TOO much scouting info???
 

Are you still sharing?

Last year at Vegas, we got 4th seed as rookies with NO scouting data and NO idea about who to pick. We made it to semis, but could have gone farther, with better info!

In fact, I'd like to see posts or links of ALL teams scouting plans and tools.

2 best pieces of advice I've seen so far are:
1. Collect PERFORMANCE data, because that's all that matters, in the end (Ask the drive team what info they think is most important on the field)
2. Assign (if possible) a scout per team in the stands to get match data that can show improvement or decline over time

Re: TOO much scouting info???
 

As the Scouting Leader of our team for the 2005-2006 year, I cannot put more stress than anything on two factors: Communication between the drive team and the scouts, and the quality of people scouting.

One of my goals this year was to analyze our scouting system last year, and over the past years to look for potential improvements and put them in place. Here are a few of the things I've noticed, in support of statements above.

1) No matter how much info you can collect, your drive team doesn't need it all. They are your drive team because they have proven that they can think quickly and make informed and fruitful decisions for the ultimate purpose of success. Our job as scouters is to give them what they want and when they want it, and not hinder them with unnecessary information. Thus, I am of the belief that you can definitly have a large amount of extra information.

2) Your scouters must be mature and committed individuals. Scouting is not just a "thursday-friday-saturday" commitment with fun and games in between. If you want your team to be prepared and efficient during competition, you as a scout must put in effort. How?

• Attend strategy sessions. One of the things you find out from teams is their strategical priorities and how they mesh with your own. To do this, you must understand your team's strategy.
• Do as much prescouting as possible. A reasonably accurate list of teams is available online months before competition, and small changes can be made on-site as needed. So, be flexible too!
• Brief the drive team / tech team at least weekly on your findings. Tell them about your plans, your organization, and stay updated on their strategic and structural priorities throughout the season.
• Everybody loves socializing. It's great, I like it too. But your team must come first if you are a scout. You can socialize in the evening at the hotel, or when the day's scouting work is finished (if it is) or you can meet people quickly when you scout their teams.
• Last but not least, although every single scout I know is very able-minded and capable, it is very advisable to have a mentor specifically tied to your scouting sub-team. Mentors can offer insight and guidance, as well as mediate any conflicts that arise. They shouldn't do jobs for you, but they can help you stay on top of your work and keep you from being overly stressed.

Above all, good luck! Competition is awesome. Our scouting system last year worked very well, and I would be happy to discuss with anybody interested the specifics of having a successful scouting team.

Re: TOO much scouting info???
 

Re: TOO much scouting info???
 

If you do plan to collect a lot of information during matches, try to develop a checklist for your scouts that will collect the information easily. Then have training sessions before competition. If possible, use some of the streamed regionals if you aren't competing the first weekend.

There is nothing worse than the scouts writing out notes longhand, and having them miss half a match while doing so. (What happened? I was writing stuff down!)

Re: TOO much scouting info???
 

Re: TOO much scouting info???
 

It's all good and well to use points scored as a criterea, but don't do what a lot of teams do and overlook defense when you're planning.

From what I saw, (I didn't scout) our team's scouting was messy and slow. The papers ended up in a big messy pile and was hard to read the writing at times, also the team members had trouble seeing the field. This year, binoculars and (maybe) a database (if I get around to learning acess) run on a 802.11b network with better dbi directional antennae for pit <--> stands communication. But, then we need the school to lend us ten(ish) laptops. Ideally I'd get a decent camera and film all the matches and connect it to a high capacity server/media center computer, before you mention the instant replay thread, remember, we aren't making juding calls on who's zip ties are in the triangle here, just deciding strategy, which (I think) doesn't need as good resolution.

Re: TOO much scouting info???
 

True, it's the complete picture you're looking for. I know we've run the gambit as far as good ways to characterize a team and bad ways to characterize a team - one thing I especially note is that team members like to tell you they can give you the world (if picked) and end up doing nothing on the field. Noting what they say they can do and what they actually do (and how they do it) is critical, but as many people have said already it comes down to what info your drivers have when it comes time to pick. Ebolagirl had a good point - drivers have usually seen nothing, and all your scouting data is useless without some kind of way to say, "Pick this team if they're available." Having tons of raw data from the scouts is kinda useless without a rubric for scoring that data, and then it turns into a matter of how well the rubric was designed. In the end it's all about gut feeling anyway; at least at Lone Star when it comes time to pick the third alliance member there are a large pool of qualified teams who all rank very similar - how do you know who to pick then? :ahh:

-Danny

Re: TOO much scouting info???
 

Why tape all the matches? Way too messy, unless you have a lot of tapes, and generally lacking quality to see what's going on with everyone.

The MOST IMPORTANT thing you can have is a picture of a robot. Really. With a few simple stats you'll be golden becuase pictures say a thousand words. Also usually there'll be one person in your scouting crowd who knows every team and their robot, or collectively you'll be able to do that.

Re: TOO much scouting info???
 

Re: TOO much scouting info???
 

Actually, taping your own robot can help in reviewing the match with the competition team to discuss the strategy they were using at different points. This may change how they react to a similar situation later.

Re: TOO much scouting info???
 

Nope. I see no way it is possible you can collect to much data. The more data you have available to you the more accurate your results/predictions will be.

I do however see how someone with so much data could not concentrate their efforts on the most important parts of the data, so I suggest some serious prioritizing of data.

Over at STAMP stampscouting.org we work really hard to prioritize our data so with a small team of scouters you can get the best "bang for your buck" so to speak with your scouting.

Also, using the videos NASA provides for regionals is well... crappy for scouting. They are horrible at giving a complete picture of the match and spend 30 seconds watching a downed robot squirm on the ground. It is not their fault, they are not trained like ESPN camera men who know exactly how to follow play and predict where the next major point of action will be. They also do not have access to 20 different cameras in different positions.

Re: TOO much scouting info???
 

Personally I view scouting as 2 part

1: Collecting and gathering as much relevant data as possible
2: Organizing and compiling this data into a useful format that is useable by team leaders (E.G. Drivers and such)

Yeah you can go overboard on part 1, collecting a crapload of irrelivant data, as much as you can overboard with part 2, organizing a bunch of irrilivent data that doesn't apply to your team.

There's many different ways to scout, all have their pros and cons, but it ultimately comes down to those two parts. If your team is capable of something extreme, go for it! Just be sure that you have enough manpower and time to compile all that data into something useable. A classic rookie mistake is to go overboard on the scouting and then all they have to show for it is a massive jumble of handwritten notes. Drivers can't use that :). On the other hand having someone just call stuff off the top of their head for teams and stratigies usually doesn't work too well either. Ultimately all that matters is that you take lots of data and organize it.


So can team's collect too much information? Not if they can organize it :)

Re: TOO much scouting info???
 

My prefered method was to walk through the pits first to see what was around, on the second time through, gather data and fill in sheets. After that, find some friends and compare data and then watch matches to see if my data was valid before i record it all in a database.

One of the mentors found the method to be flaud, but it let me see who was being kind and profressional (if they were honest).

That's just the method i used for four years

Re: TOO much scouting info???
 

taping your opponents? good idea, but very time consuming. the way I learned to scout is to have a pit form and a stand form. fill out these forms for every time and then have a folder bin to put all the papers in by team, very organized and you can tell each team does and how they are doing by pulling out the papers you have on them quickly.

and there is no such thing as too much info, EVER! the issue is the time:info ratio that people think means too much info