scouting 2011

[Zoughtbaj]

Just an observation:

a possible way for you to save on ink is to get rid of the full circles and replace them with either an excel-like sheet or circle-outlines.

Also, an idea I have is to have scouters mark the shape according to each shape, so that you can determine whether or not it is a logo just by looking at the scouting sheet.

A couple of things I would recommend:

-add match # for comparing with team members
-Drop "Score yes or no" since you will be recording what pegs are placed.
- Add "dead bot" This can be helpful to determine the reliability of the robot in some cases
-Instead of auto "yes OR no," try Uber-Tube "yes OR no" for better information
-On the top, you might want to add "Team Name." Sometimes, you might not recognize the numbers but instead the names. There are lots of teams at these regionals, usually at least 50, so it won't hurt.
-replace "yes OR no" areas with check boxes to save space
-For "Team color," you can probably drop the "Red or Blue" and have them write it in instead. Add underscores in front of it so they place it to the side, freeing up a little space.


Keep experimenting with it to see what seems to fit your liking. Good luck!

[mwtidd]

I like a lot of the work you are putting out. What's unfortunate is that the field management system this year is pretty sweet, but the twitter feed they released doesn't seem to take advantage of the information, which would have helped to eliminate much of the subjectiveness.

For the peg model, how are you going to compensate for pegs you can't see. Many of the larger competitions will force you to sit in areas with some blind spots (BAE specifically). Will you dedicate one scouter to each side of the field?

How many teams use laptops for their scouting vs stacks of paper? Both naturally have their pros and cons, and I'm curious about your thoughts for this game.

It also seems one concern is the subjectiveness of team's claims and how individuals perceive a robot. Do you think a rating system could over come this? If many individuals contributed their scouting to an evaluation of other robots, do you think they would average out to reasonable data, thus making it less subjective.

I'm very interested in collaborative platforms, it seems we tend to do 5x the work for the same payoff we could get by working together in a lot of cases. I think scouting is a perfect example, and maybe by working together we would get better data.

I think Patrick's (computerteen) idea is a good one. Definitely a step in the right direction. We see so many teams hand out flyers with claims of having the perfect robot, it sure would be nice to have a way to validate those claims.

[Waffles]

Sure, a lot of the information for the database may be opinion, but it helps us decide which teams we rank in order..

I mean, it's especially helpful with the numbers line up with everyone's feedback.

Right?

[I-DOG]

well i don't know if anyone has posted this yet....

You could include on your match scouting sheet the amount of distance a human player can aproximately throw the tube. Or instead of distance try dividing the field into zones like opposing alliance zone, opposing cautionary zone, oppsoing alliance middle, alliance middle, etc.

hope this helps

[xSAWxBLADEx]

Originally Posted by lineskier

For the peg model, how are you going to compensate for pegs you can't see. Many of the larger competitions will force you to sit in areas with some blind spots (BAE specifically). Will you dedicate one scouter to each side of the field?

How many teams use laptops for their scouting vs stacks of paper? Both naturally have their pros and cons, and I'm curious about your thoughts for this game.

with the "blind peg" comment it will be the same as scouting 07' you will just have "see-through" netting in your way

one person imputing info into the computer 6 with the sheets, the pros with this is we will have up to date scouting info in the pits when we take the computer or flashdrive and the person imputing the info knows as much as the head scout cause they are reading every paper, the cons are it's open to more human error and the imput person can't slack off

[KC1AJT]

Originally Posted by xSAWxBLADEx

with the "blind peg" comment it will be the same as scouting 07' you will just have "see-through" netting in your way

one person imputing info into the computer 6 with the sheets, the pros with this is we will have up to date scouting info in the pits when we take the computer or flashdrive and the person imputing the info knows as much as the head scout cause they are reading every paper, the cons are it's open to more human error and the imput person can't slack off

Ok i see 1 problem with that....you have about 5 minutes (including field reset) to enter ALL information into the computer for 6 team while still trying to watch the game. my team will have 6 paper scouters with 1 laptop and the data is simply logged in at the end of the match based on the paper sheets. the person inputting data shouldnt worry about the game, afterall, depending on your system, it could take 1 or 2 minutes each to log each team in (ours will take about 30 seconds).

[xSAWxBLADEx]

Originally Posted by computerteen643

Ok i see 1 problem with that....you have about 5 minutes (including field reset) to enter ALL information into the computer for 6 team while still trying to watch the game. my team will have 6 paper scouters with 1 laptop and the data is simply logged in at the end of the match based on the paper sheets. the person inputting data shouldnt worry about the game, afterall, depending on your system, it could take 1 or 2 minutes each to log each team in (ours will take about 30 seconds).

i think you misread our idea its the same as your teams

[KC1AJT]

Originally Posted by xSAWxBLADEx

i think you misread our idea its the same as your teams

oh ok send me a PM i think we can partner on a really advanced system

[Chris is me]

Originally Posted by Waffles

Sure, a lot of the information for the database may be opinion, but it helps us decide which teams we rank in order..

I mean, it's especially helpful with the numbers line up with everyone's feedback.

Right?

Qualitative data is valuable - but translating opinions into numbers is misleading. Every scout has vastly different ranking systems and opinions for robot effectiveness so taking statistical data based on that is potentially a very bad idea.

Numbers should be objective ...

[xSAWxBLADEx]

Originally Posted by Chris is me

Qualitative data is valuable - but translating opinions into numbers is misleading. Every scout has vastly different ranking systems and opinions for robot effectiveness so taking statistical data based on that is potentially a very bad idea.

Numbers should be objective ...

agreed!

[Sue-San]

Originally Posted by Waffles

-Create a logo?

While making my team's scouting sheet, I was debating putting this question on there. Does this mean that the robot made a whole logo by itself? If it's an alliance's effort, how can you judge just one robot on whether or not it creates a logo? (Or do you mean to say finish a logo?)

[I-DOG]

When makeing your match scouting sheet you should consider the following things:

You should eliminate any form of opinionated data.. However you shouldn't eliminate something like a comments section where the scouts can fill out there thoughts about what the robot accomplished in the match. If it was "tip-able", inconsistant, if they can only grab one type of tube, or things of that nature.

In Auto you'll want to know where the robot started, what rack it hung an uber tube on or if it hung one at all, if it even moves(in which case try to send your programmer to help them...it's called gracious professionalism).

In Tele you need to know what rack the tube was scored on( not the type of tube at all and i'll tell you why in a bit). Also, you'll need to keep a section for passing as some robots will just pass the whole match. Finally, keep in mind defense by creating a system of points for a robot's ability in that area...and one more final thing keep in mind the amount of penalties the robot got in the match and if you can what type.

And the reason why you do not have to tell the type of tube is if you know the final score and you know how many tubes were placed on each row and the amount of penalties given, then you should be able to know if they completed the row or not.

[Kims Robot]

Originally Posted by Siri

For instance, 1511 are scouting gods. Check out their full setup sometime, it'll blow your mind. But they've got a lot of experience in it and can get valuable information out of qualitative answers.

Wow this is humbling... lol. I would call 1114 scouting gods... and several other teams. We certainly put in the effort, but its all still being refined

One note to some of the comments, that is their very initial list. The database very often gets refined as they go. I can't speak to their exact plan this year, but from what I know that was the initial brainstorm. Once it is all put together, there will likely be refinements to make it more useful. It is just a starting point.

Originally Posted by Chris is me

Qualitative data is valuable - but translating opinions into numbers is misleading. Every scout has vastly different ranking systems and opinions for robot effectiveness so taking statistical data based on that is potentially a very bad idea.

Numbers should be objective ...

I've run 1511's Alliance Selection for the last 6 years... I think I posted about this before, but couldn't find it in a couple of quick searches so I will give you an honest rundown.

Yes, qualitative data is subjective, and yes, we change scouters every hour or so, so opinions can be different. But when you have a pile of match data from both practice matches and qualifiers, and 10 of the matches say the driver skill is "poor", and 2 say "good", you tend to lean towards the "poor" assumption.

Also, this data is compiled and given to the coach for each match, obviously the more matches that have been run, the better the data is.

We sit down on Friday night to do alliance selections, and I won't lie... its often a 4-5 hour meeting that doesn't end until 1am. We use the pit scouting data, robot photos, TBA, scores posted online and the output of our match scouting database to develop a ranked list. Its a setup of 5-6 laptops and a bunch of us spread out over a hotel room.

We start with this data and categorize every robot into Yes-high, Yes, Yes-Low, Maybe-high, Maybe, Maybe-Low and No. We then go through the list and rank each robot against all the others in the category, and move robots between categories as needed until we have ~30 robots fully ranked.

For the years that it is applicable, each robot is also given a role, if specific roles are needed... like in 2007, we were focused on picking a scoring robot first, then a ramp bot second. So once a scoring robot was picked, we had to pick from the ramp bots second.

It is a very manual process at the moment, and we have been working on the output of the database in past years to help speed up this process, but for now, it works, and the qualitative data fits.

Then on Saturday, I sit with a couple of other team members in the stands all morning and adjust the list for every single robot that is not a No. If teams were awesome all day Friday but then do terrible in their last couple matches Saturday, they get dropped in selection rank.

1511 is still aspiring to be closer to an 1114, 67, 365, 148, 217, etc etc... but their record does speak pretty well for them, they have been an alliance captain every single year, often multiple times, and sometimes even with some of their "not so good" robots. They spend a lot of time focusing on match and alliance selection strategy, and as their robots get better & better, I expect them to climb the tournament ranks more often.

I'm headed up to visit them this weekend and Im excited to see how they are doing With one of the students now heading up the database (which I think is now in SQL??) they may be able to release it ahead of week 1 for other teams to try.

I think the more teams share their scouting information & setups, the better we can make it and the better strategies you will see on the field Good luck all!

[Chris is me]

Originally Posted by Kims Robot

Yes, qualitative data is subjective, and yes, we change scouters every hour or so, so opinions can be different. But when you have a pile of match data from both practice matches and qualifiers, and 10 of the matches say the driver skill is "poor", and 2 say "good", you tend to lean towards the "poor" assumption.

Interesting. I've always thought that stuff like this would show up in the other stats, i.e. if a robot was badly driven it would score less tubes.

I know you guys have awesome scouting (I vaguely remember having a qualifying match with your team in 2009 that went very well because of it) so thanks for the detailed post explaining your methodology.

Scouting is what makes a good team great!

[jwfoss]

I can attest to the scouting methods of 1511 as they were kind enough to "adopt" me for Championships last year as my team at the time did not qualify. It was a great experience to see how they do things and to help out where I could.

I believe the real strength in their system, beyond the technology, comes from their student’s high level of involvement and understanding of each year’s games. To me, having a continuous string of students who are really into this part of FIRST, and have experience is the key to being a "strategic" team.

Be sure that the younger students are putting in the work as well and you will maintain a solid system.

2168 is currently developing our scouting/strategy system as this is something completely new to the team this year.

We are developing a heavily programmed excel worksheet that takes inputs from one “data entry” person, and six scouts (one for each robot on the field) filling out paper sheets. The data will be analyzed and used for qualification matches as well as alliance selection and understanding the abilities of our opponents. We hope to post up our excel based system before the Week 1 Regionals.

Thanks again to 1511 for letting me hang out with you guys last year.