Poor team scouting methods?

[Dan Petrovic]

Objective scouting sheets that require no additional thought for the scouters are best. Yes or No, tallies, etc.

Also, to reduce the number of sheets you have floating around, try and get as many matches on to one sheet per team. Try avoid having one sheet per match as things get messy with that many sheets of paper not to mention how many copies you have to make. ~80 matches x 6 for each robot on the field as opposed to ~40 sheets, one per team.

Then you'll want a nice way to organize these sheets. Our experience showed us that accordion folders are not the best idea. If you're in a rush, accordion folders aren't the easiest to use. We like to use huge binders with slash folders.

I hope that helps.

[Gboehm]

We took the Program for SVR with all the teams in there, and wrote comments next to them then made a list based on those comments. Cheap and effective.

[jayjaywalker3]

Originally Posted by InfernoX14

Objective scouting sheets that require no additional thought for the scouters are best. Yes or No, tallies, etc.

Also, to reduce the number of sheets you have floating around, try and get as many matches on to one sheet per team. Try avoid having one sheet per match as things get messy with that many sheets of paper not to mention how many copies you have to make. ~80 matches x 6 for each robot on the field as opposed to ~40 sheets, one per team.

Then you'll want a nice way to organize these sheets. Our experience showed us that accordion folders are not the best idea. If you're in a rush, accordion folders aren't the easiest to use. We like to use huge binders with slash folders.

I hope that helps.

I agree with objectiveness but I also want to point out that subjectiveness is also useful especially when you know who made the comment. Then you can talk to them about what they saw and it brings in a whole other dimension to a robot.

I also agree about having one sheet per team but the way I kept it organized is by using one scouter to just organize the sheets and hand out the ones we would need. This was very very effective. That person would also collect the completed sheets and sort them back into the pile of team sheets

EDIT

Originally Posted by lukevanoort

e found it pretty useful; before a match, we could tell our partners the best way to defend based on their drivesystem and how it stacked up with our opponents,
.

Can you tell me a bit about this please. I dont know enough about these drive trains.

Also nice icon InfernoX14

[lukevanoort]

Originally Posted by jayjaywalker3

Can you tell me a bit about this please. I dont know enough about these drive trains.

I can try, but it requires a pretty extensive knowledge of drivetrains. At the most basic level, we can compare coefficients of friction and determine if a robot can push another, and if so, how easily. Beyond that, it gets a bit more complicated. I've been really into drive system design since I was a sophomore, so by now, I can look at a drive system's specs and pretty accurately predict how it will preform, which is a skill gained primarily by reading every thread (2001 to present) on CD about drive systems, reading every whitepaper I can find on drive systems, as well as practical experience. Spending a lot of time running the numbers on theoretical drive system designs helps too.

Some basic rules of thumb (these don't apply in all cases, but are true in most): if the robot uses a 4WD with traditional wheels (no onmis) and their wheelbase is significantly longer than their track width, their turning will be bouncy and the robot will not be very maneuverable. If it has a 6WD with a lowered centre wheel, tapping the corner should spin it relatively easily. If a robot is using IFI traction wheels and is has a one-speed gearbox geared to go really fast (like 11+ft/s), the breakers on their drive motors will pop readily in pushing matches. Mecanum, kiwi, and other omni designs (not including swerves) tend to be really easy to push out of your way. And so on.

Basically, by using my knowledge of drive systems, we can advise alliance partners on how to defend against our opponents by exploiting the weaknesses in their drive systems (unfortunately, swerve drives have no weaknesses to exploit...), while capitalizing on the strengths of the defender's own. You could probably do something similar for counter-defense defense, but that is a bit harder to plan. Anyway, this year, such strategizing is not very useful given the restrictions on defense. You'd have better luck exploiting the weaknesses of a robot's gripper design.

[MegaSparks]

Pit scouting has its place and can be useful if the right information is gathered about the team and the robot. The pit scouting is where the vital first impression is made. I agree robot performance on the field can't be gained with pit scouting but you can see how the robot works and how it was built.

We scout matches two different ways. We get the raw stats from each match on Friday and put this into excel. The speard sheet I set up weights the different types of scoring and the totals the points for the robot. I saw a spread sheet that incorporated averages into it and I will update my spread sheet to do the same as it was a real easy way to rank the robots.

The second way we rank the robots is a more objective look at the robot. 3 or 4 scouts watch the matches and determine a score on how effective the robot was at playing the game.

These two scores are combined as a multiplier and then the teams are ranked. We have been fairly good at getting the top teams ranked correctly. It is the teams that reside in places 16 - 24 that we have a hard time with, but the multiplier works to clear that up. I am hoping the averaging method will also help clear up those middle robot rankings.

[math4origami]

I agree with a previous statement of having someone watch all the matches, while taking a few notes. Qualities such as driver skill, defensive skill, and driver choices (knock down a ball, run a lap, or play defense, or hurdle a ball) are all hard to quantify in numbers, especially if it's done by different people. It's much easier to compare robots if that one person has seen all of them preform and can compare their actions and decisions.

[platypus]

As a former scouting lead and general strategy guru, I can speak to this.

In my experience, objective evidence is king. Subjective information, regardless of the source, is riddled with issues. My reasons for avoiding subjective information:

1. Scouts form biases early and often. The number of times I've looked over the objective data and realized that my thoughts or someone else's are inconsistent with the data is astounding, and it's often due to the fact that a scout will see a team do well early and then over-credit that team for later alliance successes. The same concept works in the reverse (over-crediting teams that strike scouts as bad with later alliance failures), but to a lesser extent.

2. Opinions differ scout to scout. Everyone is rubbed differently by what they see in a robot. Period. Anything that is debatable as to whether the robot in question is at fault or another robot will have multiple opinions. Whose opinion do you trust? Any of them? None of them? What makes one scout more trustworthy than another? How do you determine any that objectively? See my point?

3. Some scouts are smarter/more experienced than others. Hence, point #2. Good luck determining how much intelligence/experience helps an individual scout - it differs from person to person.


Ultimately, subjective scouting often gives you more issues than helpful feedback. Sometimes you'll get lucky with it, but most of the time, these issues will bias your data and taint your results. Don't risk it.

Thus, I reiterate what has been said in favor of objective scouting. Go by the numbers. Crunch them in excel Friday night. If you can, set up a software database to do it for you, but if your team lacks the resources to put all of its scouts on laptops (very few teams can), then take the data on paper and enter it into the database on one laptop.

[kborer22]

Originally Posted by platypus

As a former scouting lead and general strategy guru, I can speak to this.

In my experience, objective evidence is king. Subjective information, regardless of the source, is riddled with issues. My reasons for avoiding subjective information:

1. Scouts form biases early and often. The number of times I've looked over the objective data and realized that my thoughts or someone else's are inconsistent with the data is astounding, and it's often due to the fact that a scout will see a team do well early and then over-credit that team for later alliance successes. The same concept works in the reverse (over-crediting teams that strike scouts as bad with later alliance failures), but to a lesser extent.

2. Opinions differ scout to scout. Everyone is rubbed differently by what they see in a robot. Period. Anything that is debatable as to whether the robot in question is at fault or another robot will have multiple opinions. Whose opinion do you trust? Any of them? None of them? What makes one scout more trustworthy than another? How do you determine any that objectively? See my point?

3. Some scouts are smarter/more experienced than others. Hence, point #2. Good luck determining how much intelligence/experience helps an individual scout - it differs from person to person.


Ultimately, subjective scouting often gives you more issues than helpful feedback. Sometimes you'll get lucky with it, but most of the time, these issues will bias your data and taint your results. Don't risk it.

Thus, I reiterate what has been said in favor of objective scouting. Go by the numbers. Crunch them in excel Friday night. If you can, set up a software database to do it for you, but if your team lacks the resources to put all of its scouts on laptops (very few teams can), then take the data on paper and enter it into the database on one laptop.

i have been doing scouting at events for 6 years, and now that im in college we still use the paper, picture, general pit scouting method. Especially now its a matter of molding the students to what you want to get out of them. For example, i try to sit with them all day thursday (if time permits) and will do an active commentary about the match that's going on. So we were watching matches in AZ and right away saw that 1726 was an excellent hurdler, and then we went into a quick description of how they hurdled. Now even though they were throwing balls on thursday that did not mean they could/would on fri/sat, but you know they are a team to watch out for. And sure enough, they were one of the best teams in AZ. We had another college mentor that was injured and he was just inputting match data into an excel sheet, honestly that sheet was useless with out a description. When you encourage descriptions you will mostlikely get their honest opinion about a robot as well, since the comments usually have to be short and to the point, "amazing or Terrible" you should be able to get a feel for the team by watching matches.

To me it would be like going to buy a car and only asking if it runs and what features it has, with out actually driving it and seeing for your self what work it may need, and if it actually does what the owner says it can do (aka pit scouting) It's all about conveying to the scouts how you want the data

In 2004 as a member of MORT (founding member of their now stellar scouting team), sitting in the stands for every match and being in a position to pick we were able to find robots that were amazing for us. 281 and 122, 281 i don't remember exactly what they werwe ranked, but they did well at palmetto, but our sleeper team was 122 Nasa Knights. This machine was truly amazing and was able to do it all, we were able to see that despite their poor performance in the quals, we knew they were a strong 3rd partner. If we had shown our team rep just the "raw data" there is NO WAY we would have picked them, that data does not show potential of teams, and i think you miss out on what could be the missing peice to you championship winning alliance.

[heather]

To me it would be like going to buy a car and only asking if it runs and what features it has, with out actually driving it and seeing for your self what work it may need

After using a scouting method where we mostly used subjective information in Trenton, my team was thinking of switching over to almost entirely objective information. However this^ makes a lot of sense. Does anyone have any suggestions about the best categories to look at and best types of descriptions to use in terms of the more subjective information?

Thanks.

[kborer22]

ours consists of a hybrid mode section, and just a blank spot to write down what they did, (# of laps, knocked ball, etc.) we are going to add a column for # of penalty points.

Then we have a small section for Drive Train that we would use to pit scout, (type of drive system, #of wheels/motors,speed) and space for comments on how it actually performs( fast, decent speed, slow, trouble turning, easily pushed etc.)

And then for Herding, hurdling, knocking ball each have a y/n choice to circle and then we have "How?" and the "comments"

"how" is a description of what mechanism they used, and then how effective.

If you want pm me with your email and i will send you a copy of what we use

[rogerlsmith]

Remember, anyone can go to:
http://www.firstobjective.org

Take 10 seconds and sign up, then pick My Team.

Once you are on My Team, pick 2008 Team Tool.

You can enter any FRC team number and get a nice report of their performance for this year.

If you have a mobile phone that can access the Internet, point it to:
http://www.firstobjective.org/myteam/teamdata.wml

This will give you the same info right on your phone.

AND IT'S ALL FREE!!!

Roger

[Uberbots]

our scouters are amazing at what they do. They have a system set up of 5 scouts and 1 head scout, who are each assigned to watch one robot during a single match. They record various characteristics... like driver experience, how much 'fumbling' they do while picking up the ball, average hurdling/lap times, amount of defense played on them, etc. our goal with the scouting thing is to pick out the best robots from the crowd, so if a good team gets paired with an incompatible alliance against a dominating alliance, the one good team will still get recognition amongst our scouters.

Its a neat system, and all without computers- its all paper, pencil, and mind!

[SunKing]

Oh too kind...

All this conversation has gotten me thinking though, about the difference between subjective and objective scouting. We've tried objective during the beginning part of last year, and my problem with that is that a robot that scored slowly with 4 ringers with no defense would register the same on a sheet as a robot that scored 4 under heavy defense. One's clearly better, but not according to the sheet.

Then again, at the Regionals this year though, I did see evidence of a lot of scouter bias. I guess once people make up their minds about a robot, it takes a lot to make them change their minds.

I think in the end I'd prefer a subjective system operating under a skilled, trained scouting squad. We'll have to see in Atlanta.

Speaking of Atlanta too, if any teams want to stop by Team 1124 in the crowds (look for the blue superman shirts), our scouting team would love to share our data.

[smurfgirl]

We have considered electronic scouting... but our scouters like their paper method. Here's what we do:

Pits: Two students go through the pits, stopping for each team. They take pictures of the robot so we can remember it, recognize it, and see it if we need to. They also talk to the team members about how it works, what it can do, etc. Sometimes you find out that teams have other capabilities that might not always come out in a match. You can ask what they think works the best/worst on their robot. Whatever questions you see fit. We have a sheet that identifies key characteristics. You can figure out what you want to know about your partners, and put it on your sheet.

Matches: We have six students watch each match, one for each team. They record how the robot played during the match. For example, if a robot hurdled twice, then flipped over, that sends a different message than a robot that hurdled twice and then never got control of the ball again because their partners had it, which sends a different message than a robot that hurdled twice and then their arm broke... you get the picture. Just make sure you keep track of any necessary explanations. We also note what their hybrid mode does, how well they handle the ball, how quickly things get done, if they tip over easily, how many of each point-scoring task they can do, if they got any penalties... all the stuff you'd want to know about your alliance partners.

Then all the data for each team is kept together in a binder. We put the teams in numerical order, and then add data as the day progresses.