Scouting, is it important?
 

I was having this discussion with another teammate today, is scouting important? I'm interested to see what you guys think and I'm pretty sure for a lot of you the immediate thing is "He's crazy, how can you not love scouting!?", so let me explain myself first.

First off I'm the Drive Team Captain for my team, 2228, and I have been for since the beginning of last season. To this date I have yet to find any of our scouting information useful during competition. It's much more useful, I find, to have a few good eyes out there watching the games and then before each match I'll discuss it with them briefly and find out if there is anything they've seen that I haven't or if they think we need to watch out or don't have to worry about at all. Plus for the past several years for scouting we've made up our sheets, and constructed our database, then at competition we make up a schedule and have every team member rotate in sitting there and circling in bubbles like they're taking a test instead of enjoying themselves at the competition. I think it be much better if we just did away with scouting.

Re: Scouting, is it important?
 

I suggest that you put on your flame shield as I suspect that I am going to be the first in a long line of people who disagree with you. But that's all right, it's your opinion and you're entitled to it.

That said, scouting has been tremendously important to me as a driver. Frequently before matches, our chief scouter will come to me and tell me how consistent they are, how they score, what defense needs to be played on them, their autonomous modes, and most importantly, where they score. In short, a lot of information that allows us to plan a more accurate and specific strategy that relies on information that takes more to gather than just having a few people watching matches. For instance, this team scores here and here around the 1:15 mark consistently, so try to play defense at this point in this way.

Re: Scouting, is it important?
 

I see your point to scouting not being important, however I have to disagree. As the lead strategist for 548 I have found that assigning a few robots to a student to know inside and out helps a lot. Going to your alliance partners and knowing what robot works best in which zone helped out a ton. Also guessing at your opponents strategy was helpful because it told you where your alliance might need two robots or where to play defense. I also agree that some of the scouting done is worthless. Again this year, I don't need to know what kind of ball possessor a robot uses, just how well it works. That's just my observations from two years on drive team and one year of strategy.

Re: Scouting, is it important?
 

Our team has never entered a match without strategy influenced from our scouting material. Our team has developed our scouting over the years and have an advanced system to chart what a robot does and how long they play zones. It has tremendously helped our alliance selections and have made several good picks in every tournament, the most notable would be team 1058 at GSR this past year. They made it through to be the third pick by the number 2 alliance as many alliances had overlooked them based on rank the day before, our scouting data showed them to be high in ball advances and scores. Our data has been extremely effective in determining match strategy and have won many matches every season that seemed to be un-winnable, but were due to determining a solid strategy based on facts.


Just my thoughts.

Re: Scouting, is it important?
 

What you've described is also scouting. We try to do both objective scouting (like the paper sheets you describe) as well as subjective scouting where a few smart people try to figure out the teams strategy and tendencies. We scout both for strategy for the next match, and for choosing who to pick. In both cases, we use the objective data as the starting point, and then modify our plans based on the subjective data.

Just like our robot, we improve the scouting throughout the season. We usually find that at the first regional we either collect too much objective data, or not the right types of objective data.

Re: Scouting, is it important?
 

Ooh, scouting, my favorite subject! I was at a FIRST demonstration yesterday and was talking with a new member of our team, and a member of another team who said, 'You haven't told him about scouting? When I hear scouting, I think of Bethany!'
Anyways! I have been scouting captain for three years, probably the first person on our team to take scouting really seriously and invest a lot of time in it.

By the end of the first day of regionals this year, I had a personal knowledge of most teams on the field. And could spout off this memorized data to our team captain...she liked that :D

I always scout with the intention of being in a position to tell our team captain who to pick on Saturday morning. In reality, we have never been in a picking position. Scouting has, however, helped us immensely in playing from hour to hour. Before each match I will give our driver all the data I've collected on the teams we're playing with and against. Often we're the only team on our alliance with a familiarity of our competition. The same thing happens in finals, although unhappily I've sometimes known that our alliance partners [who picked us] were not as strong as they could have been, or had unwisely chosen our third partner. [Which they would not have done had they had as good a scouting system as ours...and on it goes.]

One type of scouting that I personally attach little value to is pit scouting. [I'd love to hear other peoples' opinions on this!] Because our small team is not always able to muster anyone else to help me scout, I consider it of much more value to watch every match, than to go round to a regional with 60+ teams and talk to each one. They give me data that I hardly ever refer to: I always trust what I've seen on the field, rather than what they say [or are sure] their robot can accomplish.

One big thing I see in favor of pit scouting is building rapport with other teams. I personally enjoy meeting other team members and promoting our own team :) And that sort of relationship can pay off big on Saturday. Then there's the Friday afternoon schmoozing....but that is much easier if you already have rapport with the team in question.

So to conclude this, I'd say that if you have a small team, focus entirely on match scouting: if you have one or two extra people, send them round to the pits. Oh and try to hang out with other teams after hours/during lunch. Although I am always busy every minute with compiling the data....can I get an 'Amen!' for a bigger scouting team?

Re: Scouting, is it important?
 

This is actually how I felt all the way up until this year. At Palmetto, one member on the drive team would receive a text about the opponents and I would know pretty much how to handle the match from that.

But when I got to worlds and I saw what some of the other teams were doing, that was intense. For example, we played with team 217, and they had a folder of information for each team. We were able to sit down and develop a game plan without leaving the pits. And then another team had an entire excel database, which was pretty awesome.

I think that if you can scout, and you can scout well, then it will be to your advantage. But if your scouting information is lacking, or the people are not focused, then you'll find the data might not be worth while.

- Sunny

Re: Scouting, is it important?
 

Scouting is very important. What you described, a few good eyes watching and then talking to the driver, actually is scouting; no one said it actually has to be on paper. What our team does, among many, many other things, is the team members scout, then the scouting group and the scouting coordinator write up a strategy. Our scouting appears to be quality, as various other teams come to us and ask to use our information. (ask Bjenks548, he will tell you). In fact, last year (Lunacy) there were two separate matches in which we beat a highly qualified team (you should be able to guess), and our drivers attributed it entirely to scouting.

Re: Scouting, is it important?
 

I like to say that there are three factors in the recipe for success at a competition: a good robot, a good drive team, and a good strategy.

The third component, the strategy, relies heavily on scouting. Scouting is incredibly valuable to strategy, alliance partner selection, and success in match outcomes. You can have a fantastic bot and drivers who know how to operate it, but if you don't know how to work with your partners or how to counteract your opponents, you won't be reaching your maximum potential as a team or as an alliance.

If you find that your scouting data isn't valuable to you, it probably means that you're just not taking down the right information. Think about teams' robots in terms of their strengths and weaknesses, how they could complement your own team or why they would not do so. Look at trends in their performance over the course of a competition- are they getting better or getting worse? Have they had technical difficulties and have they resolved them? Some more qualitative observations could help to round out your quantitative data and give you something more meaningful to work off of.

(For example: just because a team scores an average of 12 times a match doesn't mean you should pick them as an alliance partner- if you both score can only score from the near zone, you might not be the best match. However, if they can score 12 times in a match and they complement your team's playing style and abilities, they're a great pick! Even if you're up against powerhouses, look for their vulnerabilities and how you could stop them- can they only score from a certain angle or position? If it takes them a long time to line up with the goal in order to score, you could try bumping their corner to misalign them and cause them to take up more valuable time. The exact things you want to watch out for vary by game, but you should get the picture.)

Re: Scouting, is it important?
 

Scouting is the most important thing a team can do at competition. At first you might think it's just useful for alliance captains but there are so many times before and during eliminations where good and up to date data will either save your butt or give you an edge. I think what really separates the great teams from the greatest teams is their scouting efforts.

If your team doesn't care about accurate scouting and you've never regretted a second round pick, you've been incredibly lucky. Luckily my team highly values scouting and my team's head scout is a very dedicated hard worker, so we've managed to grab teams that were absolute steals in the draft before.

I have seen some great teams get completely shortchanged by lackluster scouting. Do you really want to be the team with the #1 robot in the country making stupid selections because your data is terrible?

---

edit: A note. While the responders above have many great points, and qualitative scouting is very valuable, it is always, always better to have quantitative data in addition to qualitative data. You're shortchanging your team if you have the manpower for both and only choose one.

Re: Scouting, is it important?
 

I heartily recommend that approach. One less team for us to worry about then. :p

No, seriously: smurfgirl said it best. Read her post again.

Scouting wins matches and regionals. If the data you're getting from scouting isn't helpful, change the kind of data they collect.

We have 2 kinds of scouting: Pit scouting and match scouting.

Pit scouting collects basic info about the robot: number of wheels, type of end effector, whether they have a hanging mechanism or not, stuff like that. We sometimes use that to help decide on whether to pick a team, and more often on how to work with or against a team. The scouts who do this are our 'ambassadors', the really outgoing types. They make a lot of friends for themselves and the team.

Match scouting collects performance data: goals scored, how many times they hang, which zone they play, and the like. This is collected by six students watching every match, one for each robot. All the data is entered on laptops and collected on a local server in real time. (We happily make this data available to anyone who wants it).

Match scouting, for those doing the scouting, is awful. Nobody really likes to do it, but they all understand just how critically important complete and accurate information is.

Last season we also videotaped* every match at the regional. This was a tremendous help when deciding which team would fit our strategy best. Note that how well a team actually performs (i.e., their ranking) is not as relevant as their capabilities. Example: At VCU regional, Team Blue Cheese (1086) ranked 10th I think. We didn't know them from before ever, but their actions on the field fit in perfectly with how we thought to best play and win the game. We won the regional, even though there were some higher-scoring and -performing teams we could have picked. (Only aftre we pciked them did we learn of 1086's reputation as a powerhouse team. We know why now, too).


*Does anyone actually use tape anymore? Unfortunately, our video of VCU got lost (Ethan!) and was never posted to TBA. :mad:

Re: Scouting, is it important?
 

If you add one word I agree 110%. Because when you add that one word, the statement becomes a tautology.

Try this and see where it leads you:

"I think it would be much better if we just did away with useless scouting."

Toss out what isn't helping. Redirect the freed resources (people, time, whatever) into what does have an effect. Continuously improve. Replace the illusion of scouting with actual scouting.

Actual scouting gives the drivers, designers, pit crew, programmers, everyone the information they need. One way to determine what they need is to pay attention to what they use and what they don't use. If information isn't being used because it is irrelevant, then stop producing it.

It's just common sense, but common sense often gets lost in habits. Also, common sense sometimes becomes misdirected when producers and consumers aren't "on the same page".

So, "Yes" replace any and everything that is truly ineffective with something (either on the consumption or production side) with something that is more useful. Who can argue with that?

Blake

Re: Scouting, is it important?
 

He is 100% correct. Look at the information you gathered this year and see what you actually used.

I can't tell you how many times in my years in FIRST teams have come up to me and asked me questions that are mostly useless. My favorite must be "how many wheels do you have?". Once you do this, you may find your pit scouting reduced to only a handful of questions.

The best thing you can do is look at a team as they actually play and either objectively or subjectively determine how good they are. Objectively you can find this by some sort of formula, or just something as simple as how many points does a team score (this is very dependent on the game). Subjectively you can watch a whole bunch of matches and determine how good a team is and where their weaknesses/strengths are.

I find the best overall method includes a combination of both.

Re: Scouting, is it important?
 

Look at The Florida regional from this year.

My team was sitting in 8th, we picked two teams that were way down in the rankings. We beat the #1 seed alliance, then we made it all the way to the Finals and lost due to mechanical failure. we have never "really" scouted before and we knew we were going to be playing some tough teams this year. So we decided to dedicate two of our team members to scout, and they did a good job, we picked 79 from the mid-30's and 801 from IIRC 28th. Both teams had very excellent robots, but had some bad luck during the matches. If we had not scouted so thoroughly I do not think we would have even made it past the qf's.

Re: Scouting, is it important?
 

Usefulness of scouting varies with system, and I would say it is easiest to break into 2 types; pre-match scouting, specifically for relaying to the coach before matches, and long-term for in the event of being an alliance captain at a large tournament with many teams. Our pre-match scouting usually consists of one person seeing who we are playing next, and jotting notes for what the robots' roles are in the alliance, what to watch out for. Obviously knowing "this robot is very fast at back and forth defense" etc. would be useful in advance. As for the filling in bubbles, test-taking approach: yes, this is boring, and this is long-term scouting. If you were, say, second alliance captain; by the time your second selection comes, you may not know any of the robots that are left specifically, if they weren't standouts. This is where tallying those bubbles comes in; rather than sorting through a horde of strategic notes, all the tallies give you a cut-to-the-chase.

So yes, scouting is useful-knowing what they are going to helps driveteam confidence. To reduce tediousness, scout only the later matches and get a lot of people so you don't have to do it that much.

Re: Scouting, is it important?
 

Any robot can be beaten, with the right strategy.

The problem is, you need to know that strategy.

To remove the scouting from the equation removes any chance of your knowing that strategy.

I'm going to go back to 2005. With the #10 pick, 330 picked team 503. We would not have known that they were a diamond in the rough if we had not been scouting. We were very surprised that they were still available. A few hours later, and teams 330, 67, and 503 were hoisting some largish trophies and wearing some nice, shiny new medals after winning the Championship. Without scouting, there is a very good chance that we would have been knocked out in the division eliminations.

Eliminate your scouting--and have fun being eliminated. Keep your scouting, and improve it.

When I'm scouting, I couldn't care less about the technical stuff like # of wheels, arm/lift. What I care about if I'm scouting is: What is the robot doing in automode? What is the robot doing in teleop? What is the robot doing in the endgame? How well is it doing it? Does it break down often? And finally, How well does their team interact with our team? The last item is the only one that can't really be done objectively.

I'm also going to add on to what Joe was saying. We've done pen/pencil and paper for a while; the team has recently started doing other methods. (Wiimote, or something like that, IIRC). The data is input into a computer, and the pit scouting stuff is in a different place. Friday night strategy meeting goes something like this: "How do we want to play our matches tomorrow?" "Well, teams X, Y, Z have this, that, and the other, so this might be good." "OK, now, who do we want to pick/be picked by?" *hour-long debate culminating in "Keep an eye on these X teams, and if you see anybody else, let us know. These Y teams, we don't want to pair with, but don't tell them that."*

In 2005, we had a simple system: How many tetras did a team score, and where? 2006, with game objects flying everywhere, we used a "on a scale of 1-3, how good are they at X" along with "Where do they go in automode?" 2007 returned to a how many/where scouting method, which would also work for 2010 (and 2008).

Re: Scouting, is it important?
 

Extremely well put Eric. I personally believe that scouting is about 70% of you winning the match. The other 30 is from performance and human error. This year tested the ways of scouting much more because of the various ways the robots could play. I believe it's like not just an advantage over other teams but also the best possible way to have your team succeed.

Re: Scouting, is it important?
 

hi i am 2228 scouting and strategy lead for the four years ive been on the team. The person who started this thread is the coach for my team as well. I see what he means by how scouting is useless. I have to slightly agree with him just do to not enough commiitment from people match scouting for our team. I think those of us who love scouting like me understand how criticial and important it is. I mean people tell me a team number and i know the name andf where there from and i am extremely dedicated. But i think you need just a few good eyes match scouting and do some pit scouting and youll do fine. Our scouting this year in North Carolina helped us pick two good teams 401 hokie guard and 23614 mars and we did decent in the QF. But i think youll always need scouting and is the most important thing to do at regionals

Re: Scouting, is it important?
 

At some point, you will inevitably lose a match you're supposed to win. With good scouting, you may win a match (or many matches) you're supposed to lose. Of course this assumes all else equal.

If no one on the team is into scouting or into the 2nd-regression statistical analyses that may be significantly educational, then perhaps its best you do away with scouting. In my experience the lack of scouting generally points to a lack of team cohesion and the commitment to win even with a sub-par robot, thus there are bigger issues to address.

Re: Scouting, is it important?
 

Extremely well put here! Two extremely good robots cannot just walk onto a field and play to win. For example, one very good robot who plays strictly near cannot pick another strictly near robot who is also extremely good just because they score tons of balls, you will just end up with a really bad mid zone robot with a good near zone robot getting torn apart.

One thing our team added this year was time in each zone to determine where a robot likes to play.

Re: Scouting, is it important?
 

Joe got what I was getting ready to type spot on.

We learn what to look for by watching a week one regional and make our first rev of our scoutning plan. This is heavily influenced by our stratagy team deciding what we need to know to pick and what we need to know to win the match. If we learn after one event we didn't collect the right info we update our plan and move on.

All the info on each team gets filed into "the bible" after each match we watch. Our qualitative scouters work with the drive team to create stratagies that give us the best opportunity to win the match.

It's important not to just stick people as scouts, they need to want to do it and understand what it does for the team. Our group of 7-10 scouters always has a leader who wants to be there and keep things going because they know our method of scouting has done well for us over the years. If the scouts don't buy into the system and the plan then you get garbage output. Make sure that they feel they are contributing and not being wharehoused for the event.

Re: Scouting, is it important?
 

This is one of the biggest understatements I have ever heard. 177's scouting system is both fabulous and incredibly comprehensive. And if you look at their track record... well, their consecutive appearances on Einstein for the past five years and their two Championship wins are definitely heavily correlated to their excellence at scouting.

This is the part where I want to jump in with one of my favorite FIRST stories about how 1124 learned everything we know about scouting and countless other lessons from Team 177 back in 2006. We didn't really understand the purpose of scouting then, nor did we do much of it. We had some bad luck in our matches, and were ranked at 40/40 teams for a while on Saturday morning, but we moved up a few places to 34/40 after our final qualification match. We figured that with a number that low, there was no way anyone would ever pick us, and it was over for that competition. However, this was not so- after pairing up with Team 176, Team 177 invited us to join their alliance. And boy, had they done their scouting homework. In strategy discussions among the alliance, they shared information about other teams that blew our minds, because we were naive and understood virtually nothing about scouting. They knew exactly how to out-strategize each opposing alliance. To make a long story short, we played a lot of matches with the alliance because under their scouting-informed direction we kept winning them, and they opened up our eyes to the value of scouting, how to scout, and lots of other wonderful lessons that helped our team improve. (It was a bit more complicated for me; it was a magical experience for me because it was my first regional and a lot of things started to fall into place because of them... but the bottom line is still that scouting is important, and team 177 knows what they're talking about when it comes to scouting.)

Re: Scouting, is it important?
 

Having been on drive team for the first two years of my time on team 316 and then being back up drive team this past year, I have to say that scouting is a HUGE part of succeeding in competitions. If you have a strong scouting team, then each match becomes slightly easier for you. Before each of our matches this year, we had one mentor who would come down and talk to our driver and tell her what our opponents had done in their last two or three matches. She would also tell our driver whether our alliance partners had broken down during a match or not so we could prepare ourselves for a loss of an alliance partner if it happened again. Even though many of us don't like to do match scouting, it is extremely useful for our team, especially if we modify our scouting techniques to fit the specific competition.

Re: Scouting, is it important?
 

Sammy is exactly right everyone. Not going to lie, the matches where we were hectic and running around trying to talk to teams to see what they knew really hurt us. Although, there were people from the stands giving us minor hints about the robot. (Saying if it was good or bad...not really helpful haha) BUT, in those matches, I had found it much easier to talk to other teams about strategy.

So I am guessing this topic has a lock for scouting being EXTREMELY important?? :D

Re: Scouting, is it important?
 

Will scouting always be some kind of "secret weapon" that teams must do on their own?

Re: Scouting, is it important?
 

Each team is looking for something different. As such, they will want different data about the same robot.

That said, if two or three or more teams were to collaborate on scouting, nobody would look askance at it. However, the data would need to be shared equally, and the work done roughly evenly, if possible. (Or, a pre-arranged agreement made, something along the lines of, we'll provide X students to help you gather data, in exchange for Y data set that you're already collecting.)

By the way, I do know that in the past, 330 has allowed teams access to what our scouting data showed about their performance. 1114 does somewhat of the same thing by releasing their scouting report before the Championships.

Re: Scouting, is it important?
 

There are is a fantastic presentation on scouting/strategy written by Karthik that can be found on the 1114 website. I have a presentation that I gave at MN Splash that was based off of Karthik's and can be found at mnfirstregional.org

In 2009 Team 2175 collaborated on scouting at the Northstar regional with Team 171. In 2010 we again worked with 171, but a number of other teams joined in, by the end of the event the data was being shared with 10+ teams. There were definitely both pros and cons to doing things that way.

Re: Scouting, is it important?
 

Scouting is incredibly important. I mean, the information isnt always used, but we have been in numerous situations in which we have consulted with our lead scouts for advise on teams to pick. I mean, sure the first place team is usually a good team to pick. But in some games even the best robots can end up with a bad seeding. In lunacy, for instance, the game was very different, and sometimes the best robots wouldnt even be picking. Or the Uberbots this year. they had a fantastics robot, but sometimes their seeding wouldnt be as high because they would play defense sometimes. They usually ended up doing great, but the scores of a match dont always reflect the bots capabilities.

Re: Scouting, is it important?
 

While scouting is the most important thing my team does at competition, I'm not opposed to doing it with others. At WPI, team 20 was kind enough to lend us their data, while at CT we used our own with the help of sgreco on Saturday.

I do think it is 1000% understandable for any team to keep their pick list completely secret. 2791 is the kind of team that shares anything and everything, yet there are many reasons we do not make our pick list with others nor do we share it except with our first pick.

However, with regards to data collection and the way we organize information on teams, I'd be happy to share any of that with anyone. I actually have been looking into doing a pre-Kickoff presentation on the subject, since not everyone can see Karthik's wonderful presentation on scouting.

Re: Scouting, is it important?
 

This is not necessarily the case. 358 has twice given 3280 their scouting data and access to their scouts because we didn't have enough people to effectively scout on our own. In both of these situations 358 was seeded below us. In one of the two 358 was seeded below us but was also with in the top 8, meaning we were picking the robots that they wanted before they had the opportunity.

From a match prep perspective I don't think I have seen a better system than 1519. Every match I go into I feel that I have a good idea of what the other bots will do. 1519 has concrete data that shows what they will do.

Re: Scouting, is it important?
 

:o Thanks.

When it comes to sharing scouting data I don't have an issue with teams doing it. From experience I have learned there are certain teams we trust the data we get from better than others because of the type of info collected or the fidelity of the information.

There are sometimes where it's important to have teams you can trust to talk to about teams the Championship is one of those times. If you watch carefully teams get eliminated you can see people running to other fields and or to join other teams and share info.

Re: Scouting, is it important?
 

So this has got me stumped. I believe other teams when they say scouting is important (largely due to the strong correlation between those that say so and those that win). To that end, I've poured countless hours into reading references, combing through teams' databases, gathering pre-competition data, and researching and setting up a during-competition scouting method. The entire system has been through several iterations now, and we certainly know more than when we started. However, I can't figure out how to get it to the point of "indispensable". Granted, I don't think this entirely a scouting issue alone. Having been a student driver for three years and a mentor coach for one, I know there's quite a bit we lack in the way of strategy and coaching, scouting information or not. I seem to be missing the big picture as well as the smaller one.

Basically what I'm asking is, would anyone who's figured it out like to offer a specific example of what you scout for and how it's helped you determine a strategy and win a match? Or is that need-to-know only?

Tangentially: We're exceedingly short on people who enjoy scouting. How do teams recruit/motivate these people? (Or is the key just seeing their work pay off? That'd be understandable.)

Re: Scouting, is it important?
 

I'm going to go out on a limb and say that the most important thing you can do, if you only do one thing is game analysis. THis is a crucial step that many teams (ours included) fail to complete each year. For example, this year, the ball return provided a chokehold, and a gluttony of balls would appear in the middle zones. I know in our preliminary thinking, this was a key aspect of the game that was overlooked.

At the colorado regional, we were only able to bring 12 team members. This meant that pit scouting went out the door. Instead, we relied solely on an algorithm that I developed, as well as subjective observations by our scouts. We realized going into that regional that we were a striker, so we focused our efforts on finding midfielders and defenders who would complement us. We found that 1584 could clear balls all the way from the far zone with ease. Despite them losing most of their early matches, their defensive ability was notable, and that went into our system. The algorithm spat out a list of midfielders and we found a diamond in the rough with 2036.
As the 4th ranked alliance, we took down the #1 alliance on our way to victory. Many of us attribute that victory to the scouting ability, as opposed to our drivers.
Game analysis is the most important thing you can do. Find out what the strengths and weaknesses of the challenge are, and how your robot fits into them. Then develop a profile of the team that would go perfectly with you. Prioritizing this will allow you to accomplish more with fewer people.

Re: Scouting, is it important?
 

I can tell you how it hasn't. Back in Lunacy, I didn't do enough prescouting. Walk on Newton, first qual match is against 234 Cyber Blue. Any scout worth their salt would tell you Cyber Blue was fantastic last year, but I didn't have that information so the alliance strategy for that match completely ignored the giant threat they were. We lost the match and if we caught it in advance, we could have possibly turned that around.

At that same competition, we were trying to determine which partner should be the "primary scorer" and which should score occasionally and more frequently run picks and defensive plays. Both teams were convinced they had the better bot, but pulling from our scouting database we were able to objectively see who was better at scoring and not being scored on in order to properly assign roles for a crucial match.

Re: Scouting, is it important?
 

When approaching a match, the main thing you need to know is 1) what will our opponent do and 2) what can our alliance partners do. Once we get the schedule, we circle the previous match or two of each of our partners and opponents so our 'main scouts' watch them opposed to our normal, most likely freshman scouts.

Using match 33 from Midwest as an example (as it started one of the most memorable Friday afternoons as we beat 1625, 111 and 16 in four matches, ensuring us the top seed*). The match was 2338-1732-1739 VS 1625-3135-81. Our scouting data showed that 2338 was very good from the middle zone and 1739 was well driven even though it didn't have much of a kicker. The other alliance had 1625, who was a big threat, starting in the far zone and then moving up to score then hang, 3135, who was solid at pushing balls in and 81 who was having drivetrain issues. We knew that we would start in the far zone as we could kick all 3 balls in auto and 2338 would start in the middle and clear/score from there. The biggest decision was to have 1739 score from the close zone or play defense on 1625. We decided that if they could keep 1625 in the far zone (which meant blocking the tunnel as 1625 couldn't climb the bump), the alliance would struggle to score goals and also couldn't get the hanging points. Finally, we decided that 2338 would stay in the middle zone and we would move fairly quickly to the first zone and score the balls 2338 missed.

The match went perfectly as 1739 played fantastic defense, keeping 1625 in the far zone while limiting the number of balls they cleared (I think they may have scored one from the far zone). 2338 cleared enough balls and we scored enough from the first zone to win comfortably, 5-2.

I think that if you have a limited amount of scouts, I would make sure you scout your opponents and partners, as you have to win your qualification matches before you can start creating a picking list.


*We got lucky with having good partners in those matches.

Re: Scouting, is it important?
 

In our scouting prime, 1276 recorded goals scored, and where they were scored.

In 2007 we knew how many tubes were scored, and how many robots were lifted. In 2008 we knew how many hurdles teams made, and how many laps team finished. In 2009 we knew approximately how many balls teams dumped, how many balls got dumped in their own trailer, and how often they went for empty cells. We also tracked penalties every year.

To be honest, it was most useful to rate our own alliance against our opponents. We made a point to only represent our performance on the field. We wouldn't say "We can get six hurdles based on the timed runs we did at home." or even "We're pretty sure we can get 4 hurdles based on a practice match yesterday." We'd say, "We got two hurdles in every match so far. We hope to do better, but you've got at least two. Our data shows you have been averaging 2.3, and you've got 1.4. 0123 has averaged 1.7, 456 has averaged 3.1 and 789 has averaged .8."

This gives you a very objective way to assign robots to tasks. It's also helpful if their is a game breaker -- if a certain robot goes to retrieve an empty cell with 35 seconds left every time, you know someone should be there to block the hand off.

(Also, autonomous! Track them, they are probably important.)

Re: Scouting, is it important?
 

I have a request.

In the next season, I want you to ask your drivers, coaches, and any other good strategists on your team what the driver(s) needs to know about the other teams in each/any match in order to be able to make the most of your alliance's strengths (skills, machinery, and cleverness) as your alliance competes to outsmart, outmaneuver and outscore your opponents.

Ask in advance of the first competition. Ask after practicing and planning as much as you can (on paper, in simulations, using people as pretend robots, and with the real machine). Ask again during the competition.

And then have your team's scout(s) go get that information.

If the drive team asks for the info you mentioned in your post (quoted above), or for info that can be confidently derived from it; then go get it. If they don't ask for it, invest you time in getting them what they do need.

If they don't know what they need; sit down with them and collaboratively figure it out.

If they assert that learning about the other 5 robots on the field won't help them, consider forming a new drive team; or working hard to get them to the point where they can pay attention to more than just what they themselves are doing from one moment to the next.

Advice from others about what to collect is interesting; but I assert that asking your drivers/coach what they want and need is where you will find your answer.

Blake

PS: If your drivers and coach want to ask for strategy and tactics advice from other teams, and scouting suggestions to go with it, I wholeheartedly endorse getting that advice from other teams.

PPS: I am focusing on using scouting to improve you match outcomes, not on picking alliance partners. However, once you learn how to use scouting to win matches, then you will be only a short step away from to using it to improve alliance picks this coming season or the next.

Re: Scouting, is it important?
 

Allow me to enthuse. As stated many times elsewhere, I love scouting. Probably because it was my first job on the team: actually I got recruited solely because the team [consisting of 5 people] needed a scout at competition.
Strategy for the game is hopefully already a part of your build season: usually on day one. I know our team ran over several possible game tendencies on kickoff day, to decide which aspects to focus on [including the likelihood of balls being trapped in mid-zone]. Regrettably much of this strategy had been forgotten by competition. My scouting criteria [for matches] were as follows: number of balls scored, stability of robot, end-game strategy, and a few other things.

By day three I and my scouting partner down to a science: we had eliminated all the unimportant data, which proved to be everything except raw number of points scored, with the occasional note about 'hang' or 'ded'. [The other day I got overwhelmed with nostalgia when observing a scouting sheet in my partner's handwriting with the word 'ded' written down....so much faster to type that than 'dead'.]

Our strategy was probably uninventive but worked quite well: before each match I would go through our partner teams, pick out the strongest team, and set them on offense if they agreed. I would similarly look for the strongest opponent team and assign our weakest team to 'guard' them. Not necessarily to stay in the far zone. I rarely got to talk with the drive teams of our alliance partners [was rushing back to the stands in case I'd missed something] but our driver says they were usually quite willing to follow our strategy suggestions :)

I'll just say again that we did all this with two scouts. There are so many benefits to having a larger scouting team, but on the other hand that probably means that there is no one who has an in-depth sense of every team on the field. I take pride in being that person...personally I'd rather be the lone over-worked scout with the knowledge to inform our team captain, than have a program that offers up a numeric ranking for each team [as I've seen. I've also seen those programs rank a lot of teams the same which I had a far more in depth knowledge of...and I could go on and on....]

As far as recruiting dedicated scouts: they may or may not already exist on your team [it sounds like not?]. For two years I was our only scout: it was brutal [still is]. This year we had a splendid new team member, who sadly just graduated. She and I hit it off immediately and worked together for various PR endeavors such as Chairman's. We did nothing but scout all competition. She watched one alliance, took notes, while I recorded info on the other alliance on the laptop. At the end of the match all data was input to the laptop. I just used a spreadsheet for the match scouting...hope to get a less user-intensive program next year.

On Friday night I spent a few hours going through the data and recording it all under the appropriate team's heading. This took WAY too long and I hope to fix the problem next year. But by the small hours of the morning on Saturday, I had a very good idea of the top teams. We wound up getting picked in the finals: got eliminated in the QFs. But I do not think we would have gotten that far without the constant stream of data that flowed through my hands into our alliance...okay, leaving off now ;)

Anyways, recruiting dedicated scouts: I approach scouting, like everything related to FIRST, with frenetic enthusiasm and brutal dedication. You need someone like that. Keep in mind that they may not be an engineer! If they feel that their skills are not being used, they won't be their best. I'd advise hunting elsewhere: your school's DECA team, English classes, general enthused people who like the excitement that FIRST offers but may not feel that they have mechanical skills.

I'd love to expound more on this topic, feel free to ask me if you have anymore questions...

Re: Scouting, is it important?
 

Scouting starts the day after Kickoff.

When you break down the game, figure out what the possible game objectives are. Then, figure out how you're going to track them. (Meanwhile, the team should be figuring out what they'll be good at doing--that'll factor in eventually.)

Ideally, you track what is done in automode and what is done in teleop. The objectives are probably different, but you get the picture.

Now that you've got that data, it's relatively easy to filter out teams that you want. Weight the data so that the characteristics you want come to the top. That's your picklist.

Re: Scouting, is it important?
 

I assume that you mean "The day after the [previous season's] kick-off." ;)

The way I envision success, there is a collaborative process involved that deserves to be practiced (right now, in the Fall, at an off-season scrimmage, in a VRC tournament, ...) well in advance of learning any single season's game's details.

Re: Scouting, is it important?
 

I believe Eric refers to strategy as a part of understanding what information is necessary for the current year's game, rather than a team's processes or strategy for intrinsic/external collaboration. Thus that particular piece of scouting is impossible to start before kickoff. Also, in my opinion, since I know what I know about previous years' games (and students find it all too easily on the web), it's not even fun to re-derive strategies for the older games any more for the purpose of practicing strategy (and thus laying the foundations of scouting for a given historic game).

Yet I do believe what you're trying to get at is true: those who practice the elements of strategy, scouting, and collaboration in the off-season are more successful than those that do not.

Re: Scouting, is it important?
 

I personally think Day 1 is not the best time to figure out attributes your partner needs to have. I think a strategic analysis of the game will require considering what you do when your partners have this or that capability, but you shouldn't be working on scouting until you know what your robot actually does.

An example: At Kickoff, it was determined that 2791's strategy would be to be the "ultimate midfielder". We briefly (too briefly) explored a 469 strategy, emphasized our robot on having high traction, ball control, and kicking over one bump. If we decided on what to scout for then, we would have looked for a versatile defensive / kicker hybrid that could ideally climb the bump and a "striker" frontbot adept at cleaning up misses. Of course, 2791 ended up going on the field with a robot that played the front zone almost exclusively, so we would have been screwed if we had our scouting designed at Kickoff.

I think determining how to scout is one of the things you can do after ship. You don't need your robot to do it, so it's better to allocate your limited time resources to building the robot when you still have it.

Re: Scouting, is it important?
 

Wow, thanks for all the responses, everyone! This is going to take a while...

Yes, we definitely missed the boat on preliminary game analysis this year (though we've sort of managed to jump on now). Several of us have spend off-season time reviewing other games and results to hone these skills. Very good explanation, thank you. As for information gathering itself, can I ask what you looked for in making your algorithm? You don't have to explain what it actually is or anything (not that you have to expound at all). Thanks for your insight!
I'm glad pre-scouting has helped other teams (or rather, the negative). I was starting to worry it really wasn't the most effective solution for anything. I'm also glad to see others use it as a dispute settling mechanism. We're working (pretty effectively) towards this as our data improves. Great examples, thanks!

Wow, now that's an elaborate example. Thank you very much! We've started to head in this direction, I think my biggest problem now is untrained scouts. (As a drive team, we mostly know what we want to come up with a strategy like this, we just don't know how to have scouts observe & communicate it in a helpful way.)

Thanks, Curtis. I like the objectivity here. My latest version for our scouting records goals scored by zone, home zone misses, passes and missed passes, penalties and hangs. It also has a section for ball/robot/goal blocks, which our drive team finds easy to spot but our first-year scouts...plural only if we're lucky...are still learning. I did like how it was shaping up (if slowly) though, and this helps convince me we're going the correct direction. I'm glad it's working for you guys, and thanks for sharing! Note: As a drive team, we do like the subjective data as well, but that takes more experience. At this point we can only really trust our own eyes, but we're working on that.

We're actually on the same page here, Blake. Everything I've done I've done because a driver(s) or my fellow coach asked me too. I'm still working with them to figure out exactly what data we should get to effectively derive what we want. It's not so much a lack of drive team to scout coordinator communication that's our problem, though, thankfully. (That's good because I'd have even bigger issues on my hands if that were the case.) Rather, it's in the accuracy of the data and the implementation thereof that the issue seems to lie.

As far as if our drive team is indeed asking for advice from other teams: that we are, and thank you for your help. ...Granted, this isn't coincidental, considering in addition to being scout coordinator, I'm also coach and remain the most experienced driver (and drive team member) the team's ever had. In fact, that's why I took this scouting job. :) We're all curious, though.

Wow, I wish we had scouts with your enthusiasm! (And thanks for the recruiting tips.) I love this stuff too, and I'd be happy to sit in the stands all day watching matches (...if I wasn't a pit manager and a drive coach).
As far as your system, this is a very interesting perspective. Can I ask what compelled you to eliminate which zone balls were scored from, between zone passes, and/or any misses involved? At first analysis, I feel like we'd miss quite a few great defensive bots--not to mention mid zone clearers--without that data, and might overweight not-so-strong strikers. Obviously that's less than true if it's working so well for you; I'm just curious about the reasoning.
As for assigning roles, I understand the highest scorer on offense, but how did the man-to-man (err, robot-robot) D work out for you? We really haven't given that approach a thorough try (zone-D seems much more common). It's an interesting idea. Did you end up with many allies making picks/screens in your home zone? How does it handle good D robots with some scoring (mostly far zone) versus not-so-successful overall bots? Wouldn't that algorithm put the latter on defense?
Wonderful insight, thank you.

Thanks, Eric! I'm definitely starting to determine scouting metrics simultaenously with game play strategy next year. I waited much too long this year. (Granted, it's the first time we've ever even attempted anything like this.) When you say "weight the data", do you mean literally? This seems like a great idea to me--I've been messing around with the algorithm for a while--or do you find it more helpful to do it subjectively?

You bring up a good point. We've started to investigate this method, but we're aiming more towards either much older games or different competitions (VEX, VRC, etc) and surprising students with the topic. I think I'd still like to try this method, but we'll stay aware of this issue. Thanks!

Thanks, Chris; that does seem like an important qualification. What I mean (and what I thought Eric meant) is not that to set what we're looking for during build season, but to investigate how to look for it. Granted, for me this is all hypothetical until next year, but it seems like that spreads the tasks out more efficiently. I'd expect revisions after the first week of regionals/districts and probably some further elaboration later. If nothing else, it might offer yet another view of the game from analysis purposes. Good point, though.


Thanks again, everyone! EDIT: My that turned out long, sorry.

Re: Scouting, is it important?
 

Right - To borrow and paraphrase an expression from another competitive domain - Practice/train the way you want to scout; and then when the time comes, you will successfully scout the way you trained.

Acknowledging what you (JesseK) said about students being familiar with past competitions; I'm betting it will still be useful to have scouts-in-training "scout" videos of a few historical matches, or some simulated matches; and then compare their notes.

At least for myself, knowing in my head what I say I will look for and record, and actually being able to accomplish those plans as a member of a cohesive group in which each person consistely produces assessments that agree well with the others' assessments, are two different things.

Drivers need to practice driving. Scouters need to practice scouting.

Blake

Re: Scouting, is it important?
 

Chris: Of course I don't mean start figuring out what you want in a partner on Day 1. You start figuring out stuff like what you want to do, and what can be done, and how to track what you want to track. Maybe by Week 5 you know enough to figure out what you want in a partner...

Siri: I mean, assign a different weight to each characteristic. What the weights are depends on how much you need a partner to have that; i.e., a near-zone robot would rate a far-zone or a mid-zone robot higher than a near-zone robot. Penalties would be a negative weighting, if you choose to play penalty-free. The more important the quality, the higher the weighting.

Re: Scouting, is it important?
 

I did this for myself, actually. I was surprised how much I learned from watching the old videos, despite not only knowing the games and outcomes, but having driven in half a dozen competitions for each one. You see things differently (not just in the literal sense). If not to determine what you looking for, at least to determine how to look for and interpret it.
We've never really been in a position to train multiple excited scouts at once, but it seems like it'd be helpful from that perspective as well. Not just for familiarity, but to spark discussions on exactly what you're looking at and what you're getting from it. ...hmm, maybe we should utilize this more widely.
So you do mean mathematically. That's what I thought, good idea.


EDIT: hey, 100! :)

Re: Scouting, is it important?
 

In all honesty, our scouting would have been much stronger had we placed some focus on the other aspects you mentioned. We didn't, because two people cannot keep tabs on very much :o However, my scouting partner and I would jot comments: the most common were 'good defense' and 'long kick'. Those are much more subjective of course, and harder to calibrate with a larger and less experienced scouting team: although I suppose defense always tends to be a harder quality to quantify [quality to quantify? Did I just say that?]. A way we made up for not being able to keep records on which teams scored from which zone, was just to ask our team mates what they were good at. Teams tend to be more articulate when asked what they do best, rather than just 'can you score well'.

Now that I think about it, this is almost pit scouting insofar as it relies on other teams' estimates of their own abilities. I.e. asking what they are best at. I am much more inclined to trust the drive team's judgement on, say, if they can feed balls to an offensive bot or are better at close-range scoring, rather than a member of pit crew [because scouting captains are never available to talk to at their pit...] saying that 'yes we can score during autonomous'.

As far as defense: I would say it worked pretty well. But I realize now that it might not have been the best strategy at another regional. The Autodesk Oregon Regional was a week 1 event, and I have never seen so many dead and/or unhappy robots. [All Thursday I was embarrassedly telling my rookie scouting partner that 'it really does get more exciting!']. This meant that rather often our team had to take the place of missing robots, sometimes even on D. [This maybe more than anything, because of our strategy of placing weaker bots on D]. And often this meant changing zones. If a strong opponent scorer was in mid-zone trying to score, it worked better to have our D working there, and potentially being able to hand ball-control over to our offensive bot, rather than running to and fro in the far zone.

Am I ever glad we aren't a first-week event next year. I think the fact of having to deal with frequently less-than-functioning team-mates influenced our scouting strategy. Really, it was quite an event if a team scored two or three points during Quals. :P
Hope that helps, feel free to ask more :)

and Question! How heavily do most of you-all scout earlier regionals? Just watching for game trends, or actually scouting teams who'll be at your regional later? We've never had the opportunity [AOR has been week 1 for years] but this year we are a week 4 event, and I am getting enthused for watching early competitions. I'm especially interested in Seattle, which I understand is the week before AOR, because of the possibility of actually scouting teams who will be at AOR.

Re: Scouting, is it important?
 

Scouting earlier events is a tricky thing to get right. On the one hand, it can help you see how the game is played out, and what types of robots might be good this particular year. On the other hand, the webcasts are not great for scouting, and the play style will change between events.

For example: Waterloo has (or had) a reputation as a defensive regional, where you could expect damage. Other regionals are more focused on offense, like Midwest. And if your scouts choose a "weak" regional and you're playing in a "strong" regional, the game will totally change. ("Hey, guys, you know how we said that scoring 3 balls would be enough to win 75% of our matches at that event? Uhhh... We're playing against 469 at this event, and they score 6 balls, again, and again, and again, and again...")

Going with using early events as a test of your scouting is a really good idea, but take any results with a grain of salt, as things change between, say, Granite State and Los Angeles.

Re: Scouting, is it important?
 

That's an interesting trade-off to make. I like it; glad it's working for you. I'll run it by my drive team next time I'm back from college. (I'd say scouts too, but she just graduated. :eek:)

Defense - Quantifying: yes it is, that's why I'm such a fan of 1114's average defensive score numbers. They take some calibration for off-season (new drivers, etc), but if they're high you can expect them to stay there. When possible, I put a drive team member in to scout them so we sort of standardize the feedback at a useful level.

Pit Scouting: I've found this as well; people respond much accurately if you give them (preferably two) choices than just a yes-no question. I guess that does make the less information trade-off easier. Luckily, I've usually found that at least one driver/coach stays in the pit for most teams.

Defense - Style: Good point. That's about what we did at our Week 1, I guess. (when we were moving correctly...) Later it became more zone-D based on man-D (robot-D): we'd determine our zones by robot is/moves there at the start of teleop. Once we're there though, it's more about alliance coach communication, given the tradeoffs of switching zones with so many good (moving) robots. Thanks, I hadn't really thought of it that way. I think I'll flip our system around to make this more straight forward.

Early Regional Scouting: I don't know much about the western regionals, but in the east I do it with a grain of salt. Particularly, if someone does ridiculously well (i.e. wins, is 469, etc) that's something to remember. For those that don't do so well, that's to be expected, so I try not to read too much into it. We changed a lot between Week 1 and Week 4. Then again, for my Week 4 scouting, I did look at when team's regionals were. If they'd just been to a Week 3 and had also done a Week 1, I was more inclined to think they'd preform Week 3-like. A lot of teams I've found stabilize after 2 events (sometimes 3).
Another helpful aspect of early regional scouting, though, is to gain more insight into how the game's played overall. Sometimes this really is about game-breakers (Beatty, 469), but sometimes it's an overall trend or an easy change we can adapt to. I think in '08 we added/changed our ball-knocker-downers after seeing MOE (?) use theirs. To be honest, though, I get a lot of that from CD directly as opposed to match footage. Eric's right, webcasts are tricky.