Poor team scouting methods?

[EricH]

Originally Posted by squirrel

We seem to have done ok with 6 students sitting in the stands recording points scored by each robot, as well as defensive maneuvers, bonus points, autonomous/hybrid score, penalties, and whatnot. Performance seems to be far more important than anything else...so just concentrate on recording performance of the teams, if you can make a relatively simple spreadsheet Friday night that ranks them and figure out kind of which teams you'll want to pick for your alliance (if you are a relatively good seed) that should be all you need from scouting.

Pit scouting is fun because you get to see the robots and talk to the nice people....but performance on the field might be more important in the end.

Ditto for 330 (and hopefully 1135 this year), except that the spreadsheet is set up long before Friday...it's just populated then. Saturday is also important; sometimes a team has a serious breakdown--or a "nirvana" moment.

Pit scouting is used to help figure out about the team and the robot's potential characteristics.

[Nuttyman54]

Originally Posted by David

My team has found that trying to get a computer to tell you who to pick is nearly impossible (and often wrong). In the past we have tried to quatify a robots qualities, but found that having a person who watches all the matches and takes basic notes is highly effective.

It's typically a bad idea to rely only on statistics for your scouting data. Situational circumstances matter a lot, especially with the random matches. Other often less considered information such as how well a team works with an alliance and how well they respond to changes in game situations is very hard, if not impossible, to quantify, but can often make or break an eliminations alliance. I've seen many teams fall prey to this because a team that looked great on paper didn't come through in the pinch.

That one person sitting there can make all the difference...

[Brad Voracek]

For scouting, the only hard advice I can give is something that's been given many times before : a picture is worth a thousand words.

No matter how much data you collect, friday night when you are reviewing the day making a rough sketch of what picks you may make : it's really hard to remember robots by team numbers. My best advice to you is to get someone to go and take pictures, with a number in the picture, of every robot. Multiple angles if possible, trying to get their manipulator. It's always the best from my experience.

[SteveGPage]

Originally Posted by Brad Voracek

For scouting, the only hard advice I can give is something that's been given many times before : a picture is worth a thousand words.

No matter how much data you collect, friday night when you are reviewing the day making a rough sketch of what picks you may make : it's really hard to remember robots by team numbers. My best advice to you is to get someone to go and take pictures, with a number in the picture, of every robot. Multiple angles if possible, trying to get their manipulator. It's always the best from my experience.

I absolutely agree. Often, even with extensive scouting - and I think we have one of the more extensive scouting systems - there are still "holes" in the data. I make it a priority to get multiple shots of the teams on Thursday, and then any updates to the robot's configuration on Friday. That way, on Friday night when we begin to look at alliance partners, we also look at the pictures. There are more times than I can count where we saw the picture and said, "Oh yeah, they were the team that did ...." and while we have antidotal info on them, the scouting sheets and records failed to record them. With 6 teams competing every couple of minutes, there are some aspects that we see, but fail to record. Photos help "remind" you of that info.

Also, to address another point mentioned in an earlier post, I agree that performance data is much more important than pit data. We still collect a lot of pit data, but really it has a different purpose. Pit data is a good way to know who the teams are, it acts as a PR function, it gets the students involved in talking to teams, and I have found that our newest team members - usually Freshman and Sophomores - actually learn a lot about the robots when they have to ask how many CIMS, what type of transmission, Omnis vs Mecanums, etc... Secondly, it also keeps them busy, and out of trouble if they have something to do! But, while we record all that data - it is the performance data that tells us how compete in each match. What our alliance partners can do, what the opposing alliance can do, what type of strategy should we employ, etc... and then, finally, it helps us when we are then picking our alliance partners in the finals.

I tell our students that we will lose one, that we should have won, because something unforeseen will happen, but we will win one that we should have lost because of good scouting. That is our scouting mantra!

Best regards,

Steve

[MrForbes]

Originally Posted by SteveGPage

I tell our students that we will lose one, that we should have won, because something unforeseen will happen, but we will win one that we should have lost because of good scouting. That is our scouting mantra!

Same here! Thanks for providing a great explanation of the different aspects of scouting. We take pictures and put them on a laptop in the pits, we have the scouts make notes about which teams seem to be working well together on the field, and we do pit scouting (mostly to get to know the other teams). It all works together, but the performance data from match scouting seems to be most important.

We are developing the Nintendo DS scouting system because we were so impressed with the effectiveness and ease of use of 842's automated scouting system...and we realized that almost all our team members have a DS, so the hardware is mostly paid for....and our main programmer Kevin likes challenges. The real bottleneck with paper scouting is putting the data into a spreadsheet so you can use it easily. An automated scouting system (using whatever you have available or can buy) makes this a painless process, and also allows you to quickly integrate the data from Saturday morning.

[Dan Petrovic]

Objective scouting sheets that require no additional thought for the scouters are best. Yes or No, tallies, etc.

Also, to reduce the number of sheets you have floating around, try and get as many matches on to one sheet per team. Try avoid having one sheet per match as things get messy with that many sheets of paper not to mention how many copies you have to make. ~80 matches x 6 for each robot on the field as opposed to ~40 sheets, one per team.

Then you'll want a nice way to organize these sheets. Our experience showed us that accordion folders are not the best idea. If you're in a rush, accordion folders aren't the easiest to use. We like to use huge binders with slash folders.

I hope that helps.

[Gboehm]

We took the Program for SVR with all the teams in there, and wrote comments next to them then made a list based on those comments. Cheap and effective.

[ALIBI]

We go around and get a photograph of each team's robot in the pits and prepare a pit scouting sheet. Along with this we scout from the stands. Everything gets dumped into excell. We also try and prepare a brag sheet with a picture of our robot that we give to anyone that asks. If you prepare your own brag sheet, you are more than likely able to get all the information correct that the hopeful alliance partner needs. We have been known to not devulge all of our Hybrid possibilites, just to keep them guessing.

[Kristina Len]

I know that the year before I was on the team, they used paper. They would take a piece of paper and write down the scores and key points. They would also add in extra details about the robots that they thought would be helpful for the mentors and drivers in their selection of alliance members for the finals. I hope this helped.

[JackN]

I find the information gathered in pit scouting clunky and unreliable and seeing as how I run scouting for my team(s) and I have a distaste for it, we don't pit scout. We have 8-10 people in the stands with each assigned a group of teams, they record the statistical information that we want on to a master sheet for that team (each team has 1 sheet) as well as any relevant things they notice (if they are being easily defended/playing defense/how are they moving/driver ability). Before every match we get the data for each of the teams in the match (both alliance and opponent) and formulate a strategy based on what our data tells us. We then go and discuss with our alliance partners on what we want each member to achieve for that match (Defend team xxxx, place the ball at the end of the match, who handles the balls, where we want to drive). From there it is all in the hands of the drive teams to do what needs to be done.

As for my statement that pit scouting is unreliable, just because a team says they can hurdle or move at 18 fps doesn't make it true. I would rather see a team put up high numbers during the match and to see them perform on the field rather than in their shop back home.

[jayjaywalker3]

Originally Posted by InfernoX14

Objective scouting sheets that require no additional thought for the scouters are best. Yes or No, tallies, etc.

Also, to reduce the number of sheets you have floating around, try and get as many matches on to one sheet per team. Try avoid having one sheet per match as things get messy with that many sheets of paper not to mention how many copies you have to make. ~80 matches x 6 for each robot on the field as opposed to ~40 sheets, one per team.

Then you'll want a nice way to organize these sheets. Our experience showed us that accordion folders are not the best idea. If you're in a rush, accordion folders aren't the easiest to use. We like to use huge binders with slash folders.

I hope that helps.

I agree with objectiveness but I also want to point out that subjectiveness is also useful especially when you know who made the comment. Then you can talk to them about what they saw and it brings in a whole other dimension to a robot.

I also agree about having one sheet per team but the way I kept it organized is by using one scouter to just organize the sheets and hand out the ones we would need. This was very very effective. That person would also collect the completed sheets and sort them back into the pile of team sheets

EDIT

Originally Posted by lukevanoort

e found it pretty useful; before a match, we could tell our partners the best way to defend based on their drivesystem and how it stacked up with our opponents,
.

Can you tell me a bit about this please. I dont know enough about these drive trains.

Also nice icon InfernoX14

[smurfgirl]

We have considered electronic scouting... but our scouters like their paper method. Here's what we do:

Pits: Two students go through the pits, stopping for each team. They take pictures of the robot so we can remember it, recognize it, and see it if we need to. They also talk to the team members about how it works, what it can do, etc. Sometimes you find out that teams have other capabilities that might not always come out in a match. You can ask what they think works the best/worst on their robot. Whatever questions you see fit. We have a sheet that identifies key characteristics. You can figure out what you want to know about your partners, and put it on your sheet.

Matches: We have six students watch each match, one for each team. They record how the robot played during the match. For example, if a robot hurdled twice, then flipped over, that sends a different message than a robot that hurdled twice and then never got control of the ball again because their partners had it, which sends a different message than a robot that hurdled twice and then their arm broke... you get the picture. Just make sure you keep track of any necessary explanations. We also note what their hybrid mode does, how well they handle the ball, how quickly things get done, if they tip over easily, how many of each point-scoring task they can do, if they got any penalties... all the stuff you'd want to know about your alliance partners.

Then all the data for each team is kept together in a binder. We put the teams in numerical order, and then add data as the day progresses.

[lukevanoort]

Originally Posted by jayjaywalker3

Can you tell me a bit about this please. I dont know enough about these drive trains.

I can try, but it requires a pretty extensive knowledge of drivetrains. At the most basic level, we can compare coefficients of friction and determine if a robot can push another, and if so, how easily. Beyond that, it gets a bit more complicated. I've been really into drive system design since I was a sophomore, so by now, I can look at a drive system's specs and pretty accurately predict how it will preform, which is a skill gained primarily by reading every thread (2001 to present) on CD about drive systems, reading every whitepaper I can find on drive systems, as well as practical experience. Spending a lot of time running the numbers on theoretical drive system designs helps too.

Some basic rules of thumb (these don't apply in all cases, but are true in most): if the robot uses a 4WD with traditional wheels (no onmis) and their wheelbase is significantly longer than their track width, their turning will be bouncy and the robot will not be very maneuverable. If it has a 6WD with a lowered centre wheel, tapping the corner should spin it relatively easily. If a robot is using IFI traction wheels and is has a one-speed gearbox geared to go really fast (like 11+ft/s), the breakers on their drive motors will pop readily in pushing matches. Mecanum, kiwi, and other omni designs (not including swerves) tend to be really easy to push out of your way. And so on.

Basically, by using my knowledge of drive systems, we can advise alliance partners on how to defend against our opponents by exploiting the weaknesses in their drive systems (unfortunately, swerve drives have no weaknesses to exploit...), while capitalizing on the strengths of the defender's own. You could probably do something similar for counter-defense defense, but that is a bit harder to plan. Anyway, this year, such strategizing is not very useful given the restrictions on defense. You'd have better luck exploiting the weaknesses of a robot's gripper design.

[MegaSparks]

Pit scouting has its place and can be useful if the right information is gathered about the team and the robot. The pit scouting is where the vital first impression is made. I agree robot performance on the field can't be gained with pit scouting but you can see how the robot works and how it was built.

We scout matches two different ways. We get the raw stats from each match on Friday and put this into excel. The speard sheet I set up weights the different types of scoring and the totals the points for the robot. I saw a spread sheet that incorporated averages into it and I will update my spread sheet to do the same as it was a real easy way to rank the robots.

The second way we rank the robots is a more objective look at the robot. 3 or 4 scouts watch the matches and determine a score on how effective the robot was at playing the game.

These two scores are combined as a multiplier and then the teams are ranked. We have been fairly good at getting the top teams ranked correctly. It is the teams that reside in places 16 - 24 that we have a hard time with, but the multiplier works to clear that up. I am hoping the averaging method will also help clear up those middle robot rankings.

[math4origami]

I agree with a previous statement of having someone watch all the matches, while taking a few notes. Qualities such as driver skill, defensive skill, and driver choices (knock down a ball, run a lap, or play defense, or hurdle a ball) are all hard to quantify in numbers, especially if it's done by different people. It's much easier to compare robots if that one person has seen all of them preform and can compare their actions and decisions.