scouting 2011
 

hey, i'm trying to figure out what important information i should include in a scouting form for this years game. what would you guys recommend putting into this form and how would you recommend weighting the information?

thanks in advance!

Re: scouting 2011
 

What are you looking for in an alliance partner?

What would you want to know about your opponents before a match against them?

Re: scouting 2011
 

You have to think of which robot is most compatible with the possible strategies for you. I would look out for defense ability, ability to control the field, tube scoring ability, speed in traveling the field, and minibot deployment speed.

Re: scouting 2011
 

It'll be hard, but try to calculate average return time. I.E. how long it takes the team to grab a tube, score it, and come back for an offensive robot.

It's hard to formulate hard data for defensive bots, because it's only in reaction to offensive plays. Try to teach your scouts how to discern between good play and poor play, and have them keep note of it. Hard data doesn't always work.

Number of tubes scored, minibot wins/ minibot speed, breaks, penalties are all going to be important this year.

When designing a scouting sheet, just imagine a game being played in your head. You'll start thinking of things you wouldn't have thought about if you were just looking at the manual.

Re: scouting 2011
 

this is cause we have an amazing head scouter (me) but we are only going to have

-robot start (outer,middle)
-auto score (yes or no)
-pegs ("X" for tube "O" for uber)
-minibot (1st,2nd,3rd,4th)

Re: scouting 2011
 

I believe there needs to be written or typed, in words, a description of the autonomous mode. The autonomous program(s) could work well with your team or against it. For example if they can score an uber-tube on the column right by the lane (has to go diagonal) I would be impressed more than by scoring by going straight. So I would weight it more at the least.

Re: scouting 2011
 

What my team has found in the past is that open response of any form is not good. Especially when you have 6 or more people scouting. My team uses schedules for scouting so we can end up with 20 different scouts. Here's how to do it:
-give options (check boxes are good)
-yes and no is always a plus

Also, on a side note:
I am currently developing a very large scouting system. It will be setup like a kiosk (hopefully) at our pit where other teams can come in, type in a number and print a report with scores, penalties, pictures, and mechanical details. If anyone would like to help out, I am currently looking for multi-user use of an OOBase database, if its possible. I will be releasing this when it is done.

Re: scouting 2011
 

Usually our team has 6 scouters, one per robot.

This year, we're looking to have an addition of someone watching analyts/feeders. <--Believe it or not, some of them will have crazy tossing abilities. XD

Obviously, there's a couple things you can determine ahead of time with pre-match sheets (aka run around the pits and ask teams what their robot does).

(There's a few cons to PRE-MATCH scouting and then competition scouting<-- it let's you know if teams made any major improvements and you get a better indication of how they work if they say they do something and don't... because when it comes down to it, it's what you see on the field.) We also take photos of every single robot at competition.

Here's what we decided for our scouting database as far as criteria goes (we'll probably share our database in the near future):

Prematch:
-Scouter name
-Match #
-Alliance color / Team #
-Starting position
-Human player position

Mini-bot:
-Attempted deployment
-->Location: (lane, mid)
-Successfully scored
-->Blocked?
-Engagement time w/ tower

Autonomous:
-Starting position
-->Stayed on tracking line?

-Ubertube
-->Attempted?
-->Scored?

Teleoperated:
-Placed a second tube on same peg?

-Create a logo?

-Pick up off floor? feeder? both? another robot?

-Alliance role (for majority of match)
-->Supplier, score, defense? (CHECK ALL THAT APPLY)

-Placement speed
-->Slow, medium, fast

-Pickup speed
-->Slow, medium, fast

-Drive skill
-->Maneuverability
-->Precision

-Feeder skill
-->Throwing distance

Something like that... our students are still working on creating the database!

Hope this helps! :)

Re: scouting 2011
 

our sheets already say that it says where they start and where they score and if we need anymore than that we can just talk to the team

Re: scouting 2011
 

50% of that is opinion...

Re: scouting 2011
 

I am unsure what you are implying with that statement but I view that list of criteria that Waffles is going with has too much baggage. I see that something like "staying on tracking line" is way too limited to provide enough use for wide scale scouting. Light sensors aren't required to make a good autonomous.

There is way too much things to fill out as that list looks intimidating. I would rather have like a box to draw robot path in autonomous and two scoring grids to show position of tubes scored. This would eliminate stuff like "create a logo" and "scored over a uber-tube"

Re: scouting 2011
 

stuff like that is mostly opinion

Re: scouting 2011
 

throwing distance and engagement time looks more like estimations of numbers than opinion. I see the point of how judging pickup and placement speed makes scouting hard since for example, the team I am in attend a Week 1 Event, and no one has any idea what is the difference between slow, medium and fast until plenty of matches have already gone by to judge others on. Perhaps an approximation of the time is needed.

I think driver-skill is plausible somewhat since it should be obvious if a robot has clumsy movements compared to fantastic ability to play defense or move across the field.

I am disputing mainly with your 2 examples of throwing distance and engagement time with tower.

Re: scouting 2011
 

How are things like distance and speed opinion? I'm sure they have criteria (such as approximate time) for what constitutes slow, medium, and fast.

Re: scouting 2011
 

Perhaps "opinion" is a harsher word for "Qualitative Observation?" Many of these do have the ability to peer into a situation and see who did the real work, and who did something other than productive.

Unless a Human Player is strong and skilled enough to throw tubes across the feild onto the lower racks (really improbable), middle racks (highly impobable), or high racks (astronomically improbable) there is no way to put a Quantitave Observation on Human Player performance. Perhaps Human Player performance is not quite as important as it was in say... Lunacy... but a good human player who can watch the game and throw the game pieces where and when they are needed is something every strong alliance should have.

Matching and cooperative Alliance Roles are undoubtedly one of the key factors when trying to create an ideal winning alliance.

These four give a solid indication to the fluidity of a teams driving or robot mechanism. Pure numbers and ratios can give a great indicator to who the top teams are, but in most cases it is not as clear cut as that and some serious in-depth research on how a team performs under pressure can make the difference between victory and defeat.

Every team has good days and bad days, matched by good luck and bad luck in their matching, and due to this numbers alone cannot make a valid decision in my eyes. For the record I believe that ratios and other match numbers are just as invaluable (to showing the top teams for alliance pickings) as qualitave info. It takes a combination of qualitative and quantitave observations to truly make sure you are picking the team you want to.

My $0.02
Happy scouting and good luck this year!

Re: scouting 2011
 

The important thing to note is to only track stats you're actually going to use. Speed is good to have - but it will be evident in scoring ability.

Here's my list as of now:

• # of Tubes on rack
• # of Tubes Misplaced (put in wrong position for a logo)
• # of Tubes Dropped
• Time on clock when Minibot scores
• # of Tubes Passed

What else would you need that's not subjective or obvious?

Re: scouting 2011
 

Don't forget that we have to have dedicated individuals spending a bit of time analyzing after the day what teams look the best by combining those two. Then if they're not the head then inform the head of their opinion.

Re: scouting 2011
 

sorry opinion was the wrong word i just think they are unneeded, with our scouting program and sheets we are trying to make them idiot proof cause yes all that stuff sounds good now but the question is when we get to competition can a person really scout all that?

Re: scouting 2011
 

I think the team has to come to an agreement before competition on how to prioritize aspects of teams. This may help with an issue of people being unable to properly scout. I don't think the team planned on having every aspect unweighted. So they probably are not expecting perfect data, but just in case an individual can notice it I guess.

Re: scouting 2011
 

ya just giving my $0.02 cause over my 5 years of scouting (being head scout and actually scouting) the easier the scouting the more useful info you get

(i know chief delphi says my rookie year is 2009 but my sister was on the team in 2005 and we were short on scouters...then i got hooked :/)

Re: scouting 2011
 

Make a form that includes two slots for your too alliance parterns and three slots for your opponents. Put all the information that is important to the game such as if their robotics is offensive or densive, effective or ineffective, etc. Also include a part in each that shows the autonomous and a simple sketch of their robot. Once your done scouting the teams that are in your match, you should give it to the drive team, and have them evalute it. Having all the information on one sheet is much easier than having 5 sheets. :D

Good luck Scouting! :]

Re: scouting 2011
 

Here's my scouting sheet. I don't know if we'll use it, but I think it's pretty good.
I also attached my "how to use the sheet" guide because I think it's necessary in order to understand the sheet's workings.

Re: scouting 2011
 

our prototype scouting sheet this year :)

(pegs "X" for tubes circle it for uber)

Re: scouting 2011
 

Sketching the robot might be a bit of a hassle since you are in the stands and the robot might not show up until the last second so you have little time.
I would rather go to their pit after they finished a match or before eliminations/qualifications start and talk to the team.

Re: scouting 2011
 

while going over your teams scouting sheet i saw team scores (why do you need that when its in the pits 24/7)

Re: scouting 2011
 

If they're a defensive bot, it helps to know the final score because it can be a decent way to gauge how well they did. Plus it only takes like 3 seconds to look at the screen and write down the numbers.

Re: scouting 2011
 

i just feel its a waste of trees...YOU TREE KILLER :( lol

Re: scouting 2011
 

Haha. It stays on one page, it's more of a waste of ink, if anything.

Re: scouting 2011
 

ya we have 6 sheets on one page, its cramped but it fits

Re: scouting 2011
 

I just want to mention that your setup depends on your scouters as well as your strategy, drivers and robot. For instance, 1511 are scouting gods. Check out their full setup sometime, it'll blow your mind. But they've got a lot of experience in it and can get valuable information out of qualitative answers.

Some teams (mine for instance) are still building their scouting base and don't have anywhere near as experienced observers. Thus I tend to lean my scouters towards more quantitative metrics to help with comparison. That's not to say the quantitative equates to inexperience, of course. (If you believe that, you need to meet a man named Karthik.)

As far as format, my best advice is try it. Especially if you're not a week 1 competition, take your best guess and then try a dry run (with video and/or imagination). Be flexible, take feedback from both the scouters and driver team. Another one of those iterative processes.

Re: scouting 2011
 

Just an observation:

a possible way for you to save on ink is to get rid of the full circles and replace them with either an excel-like sheet or circle-outlines.

Also, an idea I have is to have scouters mark the shape according to each shape, so that you can determine whether or not it is a logo just by looking at the scouting sheet.

A couple of things I would recommend:

-add match # for comparing with team members
-Drop "Score yes or no" since you will be recording what pegs are placed.
- Add "dead bot" This can be helpful to determine the reliability of the robot in some cases
-Instead of auto "yes OR no," try Uber-Tube "yes OR no" for better information
-On the top, you might want to add "Team Name." Sometimes, you might not recognize the numbers but instead the names. There are lots of teams at these regionals, usually at least 50, so it won't hurt.
-replace "yes OR no" areas with check boxes to save space
-For "Team color," you can probably drop the "Red or Blue" and have them write it in instead. Add underscores in front of it so they place it to the side, freeing up a little space.


Keep experimenting with it to see what seems to fit your liking. Good luck!

Re: scouting 2011
 

I like a lot of the work you are putting out. What's unfortunate is that the field management system this year is pretty sweet, but the twitter feed they released doesn't seem to take advantage of the information, which would have helped to eliminate much of the subjectiveness.

For the peg model, how are you going to compensate for pegs you can't see. Many of the larger competitions will force you to sit in areas with some blind spots (BAE specifically). Will you dedicate one scouter to each side of the field?

How many teams use laptops for their scouting vs stacks of paper? Both naturally have their pros and cons, and I'm curious about your thoughts for this game.

It also seems one concern is the subjectiveness of team's claims and how individuals perceive a robot. Do you think a rating system could over come this? If many individuals contributed their scouting to an evaluation of other robots, do you think they would average out to reasonable data, thus making it less subjective.

I'm very interested in collaborative platforms, it seems we tend to do 5x the work for the same payoff we could get by working together in a lot of cases. I think scouting is a perfect example, and maybe by working together we would get better data.

I think Patrick's (computerteen) idea is a good one. Definitely a step in the right direction. We see so many teams hand out flyers with claims of having the perfect robot, it sure would be nice to have a way to validate those claims.

Re: scouting 2011
 

Sure, a lot of the information for the database may be opinion, but it helps us decide which teams we rank in order..

I mean, it's especially helpful with the numbers line up with everyone's feedback.

Right?

Re: scouting 2011
 

well i don't know if anyone has posted this yet....

You could include on your match scouting sheet the amount of distance a human player can aproximately throw the tube. Or instead of distance try dividing the field into zones like opposing alliance zone, opposing cautionary zone, oppsoing alliance middle, alliance middle, etc.

hope this helps :D

Re: scouting 2011
 

with the "blind peg" comment it will be the same as scouting 07' you will just have "see-through" netting in your way

one person imputing info into the computer 6 with the sheets, the pros with this is we will have up to date scouting info in the pits when we take the computer or flashdrive and the person imputing the info knows as much as the head scout cause they are reading every paper, the cons are it's open to more human error and the imput person can't slack off

Re: scouting 2011
 

Ok i see 1 problem with that....you have about 5 minutes (including field reset) to enter ALL information into the computer for 6 team while still trying to watch the game. my team will have 6 paper scouters with 1 laptop and the data is simply logged in at the end of the match based on the paper sheets. the person inputting data shouldnt worry about the game, afterall, depending on your system, it could take 1 or 2 minutes each to log each team in (ours will take about 30 seconds).

Re: scouting 2011
 

i think you misread our idea its the same as your teams :)

Re: scouting 2011
 

oh ok send me a PM i think we can partner on a really advanced system

Re: scouting 2011
 

Qualitative data is valuable - but translating opinions into numbers is misleading. Every scout has vastly different ranking systems and opinions for robot effectiveness so taking statistical data based on that is potentially a very bad idea.

Numbers should be objective ...

Re: scouting 2011
 

agreed!

Re: scouting 2011
 

While making my team's scouting sheet, I was debating putting this question on there. Does this mean that the robot made a whole logo by itself? If it's an alliance's effort, how can you judge just one robot on whether or not it creates a logo? (Or do you mean to say finish a logo?)

Re: scouting 2011
 

When makeing your match scouting sheet you should consider the following things:

You should eliminate any form of opinionated data.. However you shouldn't eliminate something like a comments section where the scouts can fill out there thoughts about what the robot accomplished in the match. If it was "tip-able", inconsistant, if they can only grab one type of tube, or things of that nature.

In Auto you'll want to know where the robot started, what rack it hung an uber tube on or if it hung one at all, if it even moves(in which case try to send your programmer to help them...it's called gracious professionalism).

In Tele you need to know what rack the tube was scored on( not the type of tube at all and i'll tell you why in a bit). Also, you'll need to keep a section for passing as some robots will just pass the whole match. Finally, keep in mind defense by creating a system of points for a robot's ability in that area...and one more final thing keep in mind the amount of penalties the robot got in the match and if you can what type.

And the reason why you do not have to tell the type of tube is if you know the final score and you know how many tubes were placed on each row and the amount of penalties given, then you should be able to know if they completed the row or not.

Re: scouting 2011
 

Wow this is humbling... lol. I would call 1114 scouting gods... and several other teams. We certainly put in the effort, but its all still being refined :)

One note to some of the comments, that is their very initial list. The database very often gets refined as they go. I can't speak to their exact plan this year, but from what I know that was the initial brainstorm. Once it is all put together, there will likely be refinements to make it more useful. It is just a starting point.

I've run 1511's Alliance Selection for the last 6 years... I think I posted about this before, but couldn't find it in a couple of quick searches so I will give you an honest rundown.

Yes, qualitative data is subjective, and yes, we change scouters every hour or so, so opinions can be different. But when you have a pile of match data from both practice matches and qualifiers, and 10 of the matches say the driver skill is "poor", and 2 say "good", you tend to lean towards the "poor" assumption.

Also, this data is compiled and given to the coach for each match, obviously the more matches that have been run, the better the data is.

We sit down on Friday night to do alliance selections, and I won't lie... its often a 4-5 hour meeting that doesn't end until 1am. We use the pit scouting data, robot photos, TBA, scores posted online and the output of our match scouting database to develop a ranked list. Its a setup of 5-6 laptops and a bunch of us spread out over a hotel room.

We start with this data and categorize every robot into Yes-high, Yes, Yes-Low, Maybe-high, Maybe, Maybe-Low and No. We then go through the list and rank each robot against all the others in the category, and move robots between categories as needed until we have ~30 robots fully ranked.

For the years that it is applicable, each robot is also given a role, if specific roles are needed... like in 2007, we were focused on picking a scoring robot first, then a ramp bot second. So once a scoring robot was picked, we had to pick from the ramp bots second.

It is a very manual process at the moment, and we have been working on the output of the database in past years to help speed up this process, but for now, it works, and the qualitative data fits.

Then on Saturday, I sit with a couple of other team members in the stands all morning and adjust the list for every single robot that is not a No. If teams were awesome all day Friday but then do terrible in their last couple matches Saturday, they get dropped in selection rank.

1511 is still aspiring to be closer to an 1114, 67, 365, 148, 217, etc etc... but their record does speak pretty well for them, they have been an alliance captain every single year, often multiple times, and sometimes even with some of their "not so good" robots. They spend a lot of time focusing on match and alliance selection strategy, and as their robots get better & better, I expect them to climb the tournament ranks more often.

I'm headed up to visit them this weekend and Im excited to see how they are doing :) With one of the students now heading up the database (which I think is now in SQL??) they may be able to release it ahead of week 1 for other teams to try.

I think the more teams share their scouting information & setups, the better we can make it and the better strategies you will see on the field :) Good luck all!

Re: scouting 2011
 

Interesting. I've always thought that stuff like this would show up in the other stats, i.e. if a robot was badly driven it would score less tubes.

I know you guys have awesome scouting (I vaguely remember having a qualifying match with your team in 2009 that went very well because of it) so thanks for the detailed post explaining your methodology. :)

Scouting is what makes a good team great!

Re: scouting 2011
 

I can attest to the scouting methods of 1511 as they were kind enough to "adopt" me for Championships last year as my team at the time did not qualify. It was a great experience to see how they do things and to help out where I could.

I believe the real strength in their system, beyond the technology, comes from their student’s high level of involvement and understanding of each year’s games. To me, having a continuous string of students who are really into this part of FIRST, and have experience is the key to being a "strategic" team.

Be sure that the younger students are putting in the work as well and you will maintain a solid system.

2168 is currently developing our scouting/strategy system as this is something completely new to the team this year.

We are developing a heavily programmed excel worksheet that takes inputs from one “data entry” person, and six scouts (one for each robot on the field) filling out paper sheets. The data will be analyzed and used for qualification matches as well as alliance selection and understanding the abilities of our opponents. We hope to post up our excel based system before the Week 1 Regionals.

Thanks again to 1511 for letting me hang out with you guys last year.