Re: Scouting, is it important?
 

:o Thanks.

When it comes to sharing scouting data I don't have an issue with teams doing it. From experience I have learned there are certain teams we trust the data we get from better than others because of the type of info collected or the fidelity of the information.

There are sometimes where it's important to have teams you can trust to talk to about teams the Championship is one of those times. If you watch carefully teams get eliminated you can see people running to other fields and or to join other teams and share info.

Re: Scouting, is it important?
 

So this has got me stumped. I believe other teams when they say scouting is important (largely due to the strong correlation between those that say so and those that win). To that end, I've poured countless hours into reading references, combing through teams' databases, gathering pre-competition data, and researching and setting up a during-competition scouting method. The entire system has been through several iterations now, and we certainly know more than when we started. However, I can't figure out how to get it to the point of "indispensable". Granted, I don't think this entirely a scouting issue alone. Having been a student driver for three years and a mentor coach for one, I know there's quite a bit we lack in the way of strategy and coaching, scouting information or not. I seem to be missing the big picture as well as the smaller one.

Basically what I'm asking is, would anyone who's figured it out like to offer a specific example of what you scout for and how it's helped you determine a strategy and win a match? Or is that need-to-know only?

Tangentially: We're exceedingly short on people who enjoy scouting. How do teams recruit/motivate these people? (Or is the key just seeing their work pay off? That'd be understandable.)

Re: Scouting, is it important?
 

I'm going to go out on a limb and say that the most important thing you can do, if you only do one thing is game analysis. THis is a crucial step that many teams (ours included) fail to complete each year. For example, this year, the ball return provided a chokehold, and a gluttony of balls would appear in the middle zones. I know in our preliminary thinking, this was a key aspect of the game that was overlooked.

At the colorado regional, we were only able to bring 12 team members. This meant that pit scouting went out the door. Instead, we relied solely on an algorithm that I developed, as well as subjective observations by our scouts. We realized going into that regional that we were a striker, so we focused our efforts on finding midfielders and defenders who would complement us. We found that 1584 could clear balls all the way from the far zone with ease. Despite them losing most of their early matches, their defensive ability was notable, and that went into our system. The algorithm spat out a list of midfielders and we found a diamond in the rough with 2036.
As the 4th ranked alliance, we took down the #1 alliance on our way to victory. Many of us attribute that victory to the scouting ability, as opposed to our drivers.
Game analysis is the most important thing you can do. Find out what the strengths and weaknesses of the challenge are, and how your robot fits into them. Then develop a profile of the team that would go perfectly with you. Prioritizing this will allow you to accomplish more with fewer people.

Re: Scouting, is it important?
 

I can tell you how it hasn't. Back in Lunacy, I didn't do enough prescouting. Walk on Newton, first qual match is against 234 Cyber Blue. Any scout worth their salt would tell you Cyber Blue was fantastic last year, but I didn't have that information so the alliance strategy for that match completely ignored the giant threat they were. We lost the match and if we caught it in advance, we could have possibly turned that around.

At that same competition, we were trying to determine which partner should be the "primary scorer" and which should score occasionally and more frequently run picks and defensive plays. Both teams were convinced they had the better bot, but pulling from our scouting database we were able to objectively see who was better at scoring and not being scored on in order to properly assign roles for a crucial match.

Re: Scouting, is it important?
 

When approaching a match, the main thing you need to know is 1) what will our opponent do and 2) what can our alliance partners do. Once we get the schedule, we circle the previous match or two of each of our partners and opponents so our 'main scouts' watch them opposed to our normal, most likely freshman scouts.

Using match 33 from Midwest as an example (as it started one of the most memorable Friday afternoons as we beat 1625, 111 and 16 in four matches, ensuring us the top seed*). The match was 2338-1732-1739 VS 1625-3135-81. Our scouting data showed that 2338 was very good from the middle zone and 1739 was well driven even though it didn't have much of a kicker. The other alliance had 1625, who was a big threat, starting in the far zone and then moving up to score then hang, 3135, who was solid at pushing balls in and 81 who was having drivetrain issues. We knew that we would start in the far zone as we could kick all 3 balls in auto and 2338 would start in the middle and clear/score from there. The biggest decision was to have 1739 score from the close zone or play defense on 1625. We decided that if they could keep 1625 in the far zone (which meant blocking the tunnel as 1625 couldn't climb the bump), the alliance would struggle to score goals and also couldn't get the hanging points. Finally, we decided that 2338 would stay in the middle zone and we would move fairly quickly to the first zone and score the balls 2338 missed.

The match went perfectly as 1739 played fantastic defense, keeping 1625 in the far zone while limiting the number of balls they cleared (I think they may have scored one from the far zone). 2338 cleared enough balls and we scored enough from the first zone to win comfortably, 5-2.

I think that if you have a limited amount of scouts, I would make sure you scout your opponents and partners, as you have to win your qualification matches before you can start creating a picking list.


*We got lucky with having good partners in those matches.

Re: Scouting, is it important?
 

In our scouting prime, 1276 recorded goals scored, and where they were scored.

In 2007 we knew how many tubes were scored, and how many robots were lifted. In 2008 we knew how many hurdles teams made, and how many laps team finished. In 2009 we knew approximately how many balls teams dumped, how many balls got dumped in their own trailer, and how often they went for empty cells. We also tracked penalties every year.

To be honest, it was most useful to rate our own alliance against our opponents. We made a point to only represent our performance on the field. We wouldn't say "We can get six hurdles based on the timed runs we did at home." or even "We're pretty sure we can get 4 hurdles based on a practice match yesterday." We'd say, "We got two hurdles in every match so far. We hope to do better, but you've got at least two. Our data shows you have been averaging 2.3, and you've got 1.4. 0123 has averaged 1.7, 456 has averaged 3.1 and 789 has averaged .8."

This gives you a very objective way to assign robots to tasks. It's also helpful if their is a game breaker -- if a certain robot goes to retrieve an empty cell with 35 seconds left every time, you know someone should be there to block the hand off.

(Also, autonomous! Track them, they are probably important.)

Re: Scouting, is it important?
 

I have a request.

In the next season, I want you to ask your drivers, coaches, and any other good strategists on your team what the driver(s) needs to know about the other teams in each/any match in order to be able to make the most of your alliance's strengths (skills, machinery, and cleverness) as your alliance competes to outsmart, outmaneuver and outscore your opponents.

Ask in advance of the first competition. Ask after practicing and planning as much as you can (on paper, in simulations, using people as pretend robots, and with the real machine). Ask again during the competition.

And then have your team's scout(s) go get that information.

If the drive team asks for the info you mentioned in your post (quoted above), or for info that can be confidently derived from it; then go get it. If they don't ask for it, invest you time in getting them what they do need.

If they don't know what they need; sit down with them and collaboratively figure it out.

If they assert that learning about the other 5 robots on the field won't help them, consider forming a new drive team; or working hard to get them to the point where they can pay attention to more than just what they themselves are doing from one moment to the next.

Advice from others about what to collect is interesting; but I assert that asking your drivers/coach what they want and need is where you will find your answer.

Blake

PS: If your drivers and coach want to ask for strategy and tactics advice from other teams, and scouting suggestions to go with it, I wholeheartedly endorse getting that advice from other teams.

PPS: I am focusing on using scouting to improve you match outcomes, not on picking alliance partners. However, once you learn how to use scouting to win matches, then you will be only a short step away from to using it to improve alliance picks this coming season or the next.

Re: Scouting, is it important?
 

Allow me to enthuse. As stated many times elsewhere, I love scouting. Probably because it was my first job on the team: actually I got recruited solely because the team [consisting of 5 people] needed a scout at competition.
Strategy for the game is hopefully already a part of your build season: usually on day one. I know our team ran over several possible game tendencies on kickoff day, to decide which aspects to focus on [including the likelihood of balls being trapped in mid-zone]. Regrettably much of this strategy had been forgotten by competition. My scouting criteria [for matches] were as follows: number of balls scored, stability of robot, end-game strategy, and a few other things.

By day three I and my scouting partner down to a science: we had eliminated all the unimportant data, which proved to be everything except raw number of points scored, with the occasional note about 'hang' or 'ded'. [The other day I got overwhelmed with nostalgia when observing a scouting sheet in my partner's handwriting with the word 'ded' written down....so much faster to type that than 'dead'.]

Our strategy was probably uninventive but worked quite well: before each match I would go through our partner teams, pick out the strongest team, and set them on offense if they agreed. I would similarly look for the strongest opponent team and assign our weakest team to 'guard' them. Not necessarily to stay in the far zone. I rarely got to talk with the drive teams of our alliance partners [was rushing back to the stands in case I'd missed something] but our driver says they were usually quite willing to follow our strategy suggestions :)

I'll just say again that we did all this with two scouts. There are so many benefits to having a larger scouting team, but on the other hand that probably means that there is no one who has an in-depth sense of every team on the field. I take pride in being that person...personally I'd rather be the lone over-worked scout with the knowledge to inform our team captain, than have a program that offers up a numeric ranking for each team [as I've seen. I've also seen those programs rank a lot of teams the same which I had a far more in depth knowledge of...and I could go on and on....]

As far as recruiting dedicated scouts: they may or may not already exist on your team [it sounds like not?]. For two years I was our only scout: it was brutal [still is]. This year we had a splendid new team member, who sadly just graduated. She and I hit it off immediately and worked together for various PR endeavors such as Chairman's. We did nothing but scout all competition. She watched one alliance, took notes, while I recorded info on the other alliance on the laptop. At the end of the match all data was input to the laptop. I just used a spreadsheet for the match scouting...hope to get a less user-intensive program next year.

On Friday night I spent a few hours going through the data and recording it all under the appropriate team's heading. This took WAY too long and I hope to fix the problem next year. But by the small hours of the morning on Saturday, I had a very good idea of the top teams. We wound up getting picked in the finals: got eliminated in the QFs. But I do not think we would have gotten that far without the constant stream of data that flowed through my hands into our alliance...okay, leaving off now ;)

Anyways, recruiting dedicated scouts: I approach scouting, like everything related to FIRST, with frenetic enthusiasm and brutal dedication. You need someone like that. Keep in mind that they may not be an engineer! If they feel that their skills are not being used, they won't be their best. I'd advise hunting elsewhere: your school's DECA team, English classes, general enthused people who like the excitement that FIRST offers but may not feel that they have mechanical skills.

I'd love to expound more on this topic, feel free to ask me if you have anymore questions...

Re: Scouting, is it important?
 

Scouting starts the day after Kickoff.

When you break down the game, figure out what the possible game objectives are. Then, figure out how you're going to track them. (Meanwhile, the team should be figuring out what they'll be good at doing--that'll factor in eventually.)

Ideally, you track what is done in automode and what is done in teleop. The objectives are probably different, but you get the picture.

Now that you've got that data, it's relatively easy to filter out teams that you want. Weight the data so that the characteristics you want come to the top. That's your picklist.

Re: Scouting, is it important?
 

I assume that you mean "The day after the [previous season's] kick-off." ;)

The way I envision success, there is a collaborative process involved that deserves to be practiced (right now, in the Fall, at an off-season scrimmage, in a VRC tournament, ...) well in advance of learning any single season's game's details.

Re: Scouting, is it important?
 

I believe Eric refers to strategy as a part of understanding what information is necessary for the current year's game, rather than a team's processes or strategy for intrinsic/external collaboration. Thus that particular piece of scouting is impossible to start before kickoff. Also, in my opinion, since I know what I know about previous years' games (and students find it all too easily on the web), it's not even fun to re-derive strategies for the older games any more for the purpose of practicing strategy (and thus laying the foundations of scouting for a given historic game).

Yet I do believe what you're trying to get at is true: those who practice the elements of strategy, scouting, and collaboration in the off-season are more successful than those that do not.

Re: Scouting, is it important?
 

I personally think Day 1 is not the best time to figure out attributes your partner needs to have. I think a strategic analysis of the game will require considering what you do when your partners have this or that capability, but you shouldn't be working on scouting until you know what your robot actually does.

An example: At Kickoff, it was determined that 2791's strategy would be to be the "ultimate midfielder". We briefly (too briefly) explored a 469 strategy, emphasized our robot on having high traction, ball control, and kicking over one bump. If we decided on what to scout for then, we would have looked for a versatile defensive / kicker hybrid that could ideally climb the bump and a "striker" frontbot adept at cleaning up misses. Of course, 2791 ended up going on the field with a robot that played the front zone almost exclusively, so we would have been screwed if we had our scouting designed at Kickoff.

I think determining how to scout is one of the things you can do after ship. You don't need your robot to do it, so it's better to allocate your limited time resources to building the robot when you still have it.

Re: Scouting, is it important?
 

Wow, thanks for all the responses, everyone! This is going to take a while...

Yes, we definitely missed the boat on preliminary game analysis this year (though we've sort of managed to jump on now). Several of us have spend off-season time reviewing other games and results to hone these skills. Very good explanation, thank you. As for information gathering itself, can I ask what you looked for in making your algorithm? You don't have to explain what it actually is or anything (not that you have to expound at all). Thanks for your insight!
I'm glad pre-scouting has helped other teams (or rather, the negative). I was starting to worry it really wasn't the most effective solution for anything. I'm also glad to see others use it as a dispute settling mechanism. We're working (pretty effectively) towards this as our data improves. Great examples, thanks!

Wow, now that's an elaborate example. Thank you very much! We've started to head in this direction, I think my biggest problem now is untrained scouts. (As a drive team, we mostly know what we want to come up with a strategy like this, we just don't know how to have scouts observe & communicate it in a helpful way.)

Thanks, Curtis. I like the objectivity here. My latest version for our scouting records goals scored by zone, home zone misses, passes and missed passes, penalties and hangs. It also has a section for ball/robot/goal blocks, which our drive team finds easy to spot but our first-year scouts...plural only if we're lucky...are still learning. I did like how it was shaping up (if slowly) though, and this helps convince me we're going the correct direction. I'm glad it's working for you guys, and thanks for sharing! Note: As a drive team, we do like the subjective data as well, but that takes more experience. At this point we can only really trust our own eyes, but we're working on that.

We're actually on the same page here, Blake. Everything I've done I've done because a driver(s) or my fellow coach asked me too. I'm still working with them to figure out exactly what data we should get to effectively derive what we want. It's not so much a lack of drive team to scout coordinator communication that's our problem, though, thankfully. (That's good because I'd have even bigger issues on my hands if that were the case.) Rather, it's in the accuracy of the data and the implementation thereof that the issue seems to lie.

As far as if our drive team is indeed asking for advice from other teams: that we are, and thank you for your help. ...Granted, this isn't coincidental, considering in addition to being scout coordinator, I'm also coach and remain the most experienced driver (and drive team member) the team's ever had. In fact, that's why I took this scouting job. :) We're all curious, though.

Wow, I wish we had scouts with your enthusiasm! (And thanks for the recruiting tips.) I love this stuff too, and I'd be happy to sit in the stands all day watching matches (...if I wasn't a pit manager and a drive coach).
As far as your system, this is a very interesting perspective. Can I ask what compelled you to eliminate which zone balls were scored from, between zone passes, and/or any misses involved? At first analysis, I feel like we'd miss quite a few great defensive bots--not to mention mid zone clearers--without that data, and might overweight not-so-strong strikers. Obviously that's less than true if it's working so well for you; I'm just curious about the reasoning.
As for assigning roles, I understand the highest scorer on offense, but how did the man-to-man (err, robot-robot) D work out for you? We really haven't given that approach a thorough try (zone-D seems much more common). It's an interesting idea. Did you end up with many allies making picks/screens in your home zone? How does it handle good D robots with some scoring (mostly far zone) versus not-so-successful overall bots? Wouldn't that algorithm put the latter on defense?
Wonderful insight, thank you.

Thanks, Eric! I'm definitely starting to determine scouting metrics simultaenously with game play strategy next year. I waited much too long this year. (Granted, it's the first time we've ever even attempted anything like this.) When you say "weight the data", do you mean literally? This seems like a great idea to me--I've been messing around with the algorithm for a while--or do you find it more helpful to do it subjectively?

You bring up a good point. We've started to investigate this method, but we're aiming more towards either much older games or different competitions (VEX, VRC, etc) and surprising students with the topic. I think I'd still like to try this method, but we'll stay aware of this issue. Thanks!

Thanks, Chris; that does seem like an important qualification. What I mean (and what I thought Eric meant) is not that to set what we're looking for during build season, but to investigate how to look for it. Granted, for me this is all hypothetical until next year, but it seems like that spreads the tasks out more efficiently. I'd expect revisions after the first week of regionals/districts and probably some further elaboration later. If nothing else, it might offer yet another view of the game from analysis purposes. Good point, though.


Thanks again, everyone! EDIT: My that turned out long, sorry.

Re: Scouting, is it important?
 

Right - To borrow and paraphrase an expression from another competitive domain - Practice/train the way you want to scout; and then when the time comes, you will successfully scout the way you trained.

Acknowledging what you (JesseK) said about students being familiar with past competitions; I'm betting it will still be useful to have scouts-in-training "scout" videos of a few historical matches, or some simulated matches; and then compare their notes.

At least for myself, knowing in my head what I say I will look for and record, and actually being able to accomplish those plans as a member of a cohesive group in which each person consistely produces assessments that agree well with the others' assessments, are two different things.

Drivers need to practice driving. Scouters need to practice scouting.

Blake

Re: Scouting, is it important?
 

Chris: Of course I don't mean start figuring out what you want in a partner on Day 1. You start figuring out stuff like what you want to do, and what can be done, and how to track what you want to track. Maybe by Week 5 you know enough to figure out what you want in a partner...

Siri: I mean, assign a different weight to each characteristic. What the weights are depends on how much you need a partner to have that; i.e., a near-zone robot would rate a far-zone or a mid-zone robot higher than a near-zone robot. Penalties would be a negative weighting, if you choose to play penalty-free. The more important the quality, the higher the weighting.