Go to Post Know that CD is a safe environment to learn to talk with a foot in your mouth. - Andrew Schreiber [more]
Home
Go Back   Chief Delphi > Competition > Rules/Strategy > Scouting
CD-Media   CD-Spy  
portal register members calendar search Today's Posts Mark Forums Read FAQ rules

 
Closed Thread
 
Thread Tools Rating: Thread Rating: 2 votes, 5.00 average. Display Modes
  #1   Spotlight this post!  
Unread 29-02-2016, 11:41
BrennanB's Avatar
BrennanB BrennanB is offline
TBC is good at getting almost first
AKA: Brennan Bibic
FRC #4476 (W.A.F.F.L.E.S.)
Team Role: Mentor
 
Join Date: Jan 2012
Rookie Year: 2006
Location: Kingston
Posts: 1,292
BrennanB has a reputation beyond reputeBrennanB has a reputation beyond reputeBrennanB has a reputation beyond reputeBrennanB has a reputation beyond reputeBrennanB has a reputation beyond reputeBrennanB has a reputation beyond reputeBrennanB has a reputation beyond reputeBrennanB has a reputation beyond reputeBrennanB has a reputation beyond reputeBrennanB has a reputation beyond reputeBrennanB has a reputation beyond repute
Re: Qualitative Scouting?

Qualitative information is 100% more useful than unused or misused quantitative information.

So many teams try to scout too many numbers. You have to be careful to not mis-record/under use/draw strange conclusions from this.

Add me to the qualitative scout hype train. If given a choice between a qualitative and a quantitative system only for an average team, I would choose the qualitative 9 times out of 10.
__________________

Brennan Bibic - @b_bibic - Eh-Nalysis
10 years of inspiration compilation - W.A.F.F.L.E.S. #4476 YouTube Channel - 64 events and counting!

FRC 2013-2017 (W.A.F.F.L.E.S. Community Robotics #4476) - FLL 2006-2017 (W.A.F.F.L.E.S. #105) - VRC 2010-2013, 2015-2016 (W.A.F.F.L.E.S. #4476)
FRC 2009-2012 (K-Botics #2809)

Fantasy First 2014-2017 (The Breakfast Company) #TBCWin
2014 Dean's List Finalist

"Work until your idols become your rivals."
  #2   Spotlight this post!  
Unread 29-02-2016, 12:07
Green Potato Green Potato is offline
Registered User
AKA: Tory Farmer
FRC #0422 (Mech Tech Dragons)
Team Role: Tactician
 
Join Date: Feb 2015
Rookie Year: 2009
Location: Richmond, VA
Posts: 55
Green Potato is a glorious beacon of lightGreen Potato is a glorious beacon of lightGreen Potato is a glorious beacon of lightGreen Potato is a glorious beacon of lightGreen Potato is a glorious beacon of light
Re: Qualitative Scouting?

In my experience heading up our team's scouting department, both qualitative and quantitative scouting can be VERY helpful, but you need to address the inevitable issues before the competition:

Generally speaking, the problem with qualitative information is that different scouts have different reactions on the same event, and some form of training/ rubric is extremely helpful, even 100% necessary if your scouts are, like ours, younger team members.

If you go quantitative, you need to make sure that you can accurately interpret the data you get. If you don't attach any external meaning to a statistic you collect (shots, cycles, etc) it literally isn't a statistic any more, and looses its value as a result. Also, It's a good idea to add a "comments" area: some times, it's not obvious if action X should be counted in statistic Y, so allowing the individual scout to write it down is quite helpful.
__________________
Everything that can go wrong WILL go wrong, except the things we expect to go wrong and actually plan for.
  #3   Spotlight this post!  
Unread 29-02-2016, 12:14
evanperryg's Avatar
evanperryg evanperryg is offline
IT'S THE BUMP N' DUMP
AKA: Evan Grove
FRC #4536 (The Minutebots)
Team Role: Mentor
 
Join Date: Apr 2013
Rookie Year: 2011
Location: Minneapolis, MN
Posts: 656
evanperryg has a reputation beyond reputeevanperryg has a reputation beyond reputeevanperryg has a reputation beyond reputeevanperryg has a reputation beyond reputeevanperryg has a reputation beyond reputeevanperryg has a reputation beyond reputeevanperryg has a reputation beyond reputeevanperryg has a reputation beyond reputeevanperryg has a reputation beyond reputeevanperryg has a reputation beyond reputeevanperryg has a reputation beyond repute
Re: Qualitative Scouting?

Quote:
Originally Posted by Green Potato View Post
Generally speaking, the problem with qualitative information is that different scouts have different reactions on the same event, and some form of training/ rubric is extremely helpful, even 100% necessary if your scouts are, like ours, younger team members.

If you go quantitative, you need to make sure that you can accurately interpret the data you get. If you don't attach any external meaning to a statistic you collect (shots, cycles, etc) it literally isn't a statistic any more, and looses its value as a result. Also, It's a good idea to add a "comments" area: some times, it's not obvious if action X should be counted in statistic Y, so allowing the individual scout to write it down is quite helpful.
You've brought up a couple really good points. Any team looking to do good qualitative scouting should record who is taking what notes. We can go back and see, "okay, this person said this about this team, and this person said this about the same team." This allows us to negate the effects of scouter bias, and discussion between disagreeing qualitative scouters can bring out amazing details on a team's strengths and weaknesses. Also, track the match numbers corresponding to specific notes, this allows you to easily explain anomalies in quantitative performance.

Remember, qualitative data is helpful alone, and so is quantitative data. However, when the two are combined efficiently and effectively, you can derive much more from the data than you could with either one alone.
__________________
FRCDesigns Contributor | "There is only one corner of the universe you can be certain of improving, and that's your own self." -Aldous Huxley
2012-2016 | FRC Team 2338: Gear it Forward
2013
Wisconsin Regional Winner 2014 Midwest Regional Finalist 2015 Midwest Regional Chairman's Award, Finalist, Archimedes Division Champion, IRI Semifinalist 2016 Midwest Regional Chairman's Award, Finalist, Archimedes Division Gracious Professionalism Award, R2OC Winner
2015 | FTC Team 10266: Mach Speed
2015
Highland Park Qualifier Winner, Motivate Award
2017-???? | FRC Team 4536: The Minutebots

Thanks to the alliances and friends I've made along the way: 33 74 107 111 167 171 234 548 1023 1089 1323 1625 1675 1732 1756 2064 2077 2122 2202 2358 2451 2512 2826 3936 3996 4039 4085 4241 5006 5401 5568 5847 5934
  #4   Spotlight this post!  
Unread 29-02-2016, 12:39
P.J.'s Avatar
P.J. P.J. is offline
Proud Member of the Herd
no team
Team Role: Alumni
 
Join Date: Apr 2011
Rookie Year: 2007
Location: US
Posts: 247
P.J. has a reputation beyond reputeP.J. has a reputation beyond reputeP.J. has a reputation beyond reputeP.J. has a reputation beyond reputeP.J. has a reputation beyond reputeP.J. has a reputation beyond reputeP.J. has a reputation beyond reputeP.J. has a reputation beyond reputeP.J. has a reputation beyond reputeP.J. has a reputation beyond reputeP.J. has a reputation beyond repute
Re: Qualitative Scouting?

As many people have said, both quantitative and qualitative (or as we call it on 910, objective and subjective) scouting are incredibly useful in different ways. Both have their strengths and their weaknesses. On 910, we do both but the jobs are handled by different people.

The objective/quantitative data is gathered via paper and pencil scouting on sheets mainly involving tally marks in boxes. These sheets contain a box for notes so that these scouts can record things they notice that are not accounted for on the sheet, but this is not their primary task. This is what the majority of scouts are doing at a competition. (Anywhere from 6 to 24 people depending on how many of our team members are able to attend competitions)

The subjective/qualitative data is the responsibility of a much smaller group. These scouts use a legal pad to simply take notes on robot features, driving ability, general strategies, strengths, weaknesses, etc. These scouts earn their position, they take a test before our first competition and the head scouts and myself look over their answers and determine who will be given this position. We also factor in our observations regarding their abilities as an objective scout. In the past this group has been as small as two people but as our scouting system has become more refined and our team has grown we are lucky enough to have 6 subjective scouts this year, allowing them to each take a position (R1, R2, etc.) and focus on only one robot per match.

While I am lucky enough to be working on a fairly large team, our overall system is able to be scaled to as few as 4 people if absolutely necessary.

tl;dr In my experience both Qualitative and Quantitative scouting are important components of a successful scouting system.
__________________
P.J. Lewalski
Team 910-The Foley Freeze -- Student 2007-2010, Mentor 2012-2016
FRC Referee 2011-Present
Referee at 29 Regular Season Events, 3 World Championships, 24 Off Season Events, and Counting

  #5   Spotlight this post!  
Unread 03-03-2016, 13:09
Amy33Amy33's Avatar
Amy33Amy33 Amy33Amy33 is offline
Alumni
FRC #2846 (FireBears)
Team Role: Alumni
 
Join Date: Feb 2013
Rookie Year: 2012
Location: Minnesota
Posts: 41
Amy33Amy33 is an unknown quantity at this point
I think I had a post about this last year, qualitative scouting can be good if it's detailed, and if scouters can properly recognize their own biases. If it's not detailed, like our team's qualitative notes last year... well... *war flashbacks of team captain standing at alliance selections for 10 minutes silent*
  #6   Spotlight this post!  
Unread 09-03-2016, 17:32
wjordan wjordan is offline
FRC Stathead
AKA: Wes Jordan
FRC #2363 (Triple Helix)
Team Role: College Student
 
Join Date: Apr 2015
Rookie Year: 2013
Location: Blacksburg, VA
Posts: 40
wjordan is a jewel in the roughwjordan is a jewel in the roughwjordan is a jewel in the roughwjordan is a jewel in the rough
Lightbulb Re: Qualitative Scouting?

2363 have done qualitative scouting since 2014, and it has resulted in a massive improvement in our ability to judge the quality of the rest of the teams at the competition. We had done the standard quantitative stuff for awhile, but the numbers gathered by quantitative methods are pretty much useless if presented without context, and qualitative scouting helps to confirm trends that are presented by the statistics.

Our system in 2014-15 involved 6 quantitative scouts on Kindle Fires, a coordinator who would manage the scouts and occasionally gather the tablets up to load them into a database, and 1-2 qualitative scouts (the strategy coordinator and someone else, usually) who would take notes on the teams currently playing in the match on index cards.
  #7   Spotlight this post!  
Unread 09-03-2016, 17:38
Boltman Boltman is online now
Registered User
FRC #5137 (Iron Kodiaks)
Team Role: Mentor
 
Join Date: Apr 2014
Rookie Year: 2014
Location: San Diego
Posts: 823
Boltman has much to be proud ofBoltman has much to be proud ofBoltman has much to be proud ofBoltman has much to be proud ofBoltman has much to be proud ofBoltman has much to be proud ofBoltman has much to be proud ofBoltman has much to be proud ofBoltman has much to be proud of
Re: Qualitative Scouting?

With a smaller field in CVR coming off a successful SD scouting endeavor we plan to fine tune our qualitative techniques and get it down to a science...love qualitative scouting. It works.. I was watching the eliminations going "yup there it is". Almost like you can predict how these teams play.
__________________

Iron Kodiaks Team #5137 San Marcos, CA

2016 Semi-Finalist | Central Valley Alliance Captain #2
2016 Semi-Finalist | San Diego 2nd bot alliance #8
2015 Semi-Finalist | Ventura 3rd bot alliance #3
2015 Quarter-Finalist| San Diego 2nd bot alliance #5
2014 Rookie All-Star | #21 San Diego | Galileo Division #91
Closed Thread


Thread Tools
Display Modes Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Forum Jump


All times are GMT -5. The time now is 17:06.

The Chief Delphi Forums are sponsored by Innovation First International, Inc.


Powered by vBulletin® Version 3.6.4
Copyright ©2000 - 2017, Jelsoft Enterprises Ltd.
Copyright © Chief Delphi