Go to Post Many software problems can be solved by plugging wires in. - Eric Finn [more]
Home
Go Back   Chief Delphi > Competition > Rules/Strategy > Scouting
CD-Media   CD-Spy  
portal register members calendar search Today's Posts Mark Forums Read FAQ rules

 
Closed Thread
Thread Tools Rate Thread Display Modes
  #1   Spotlight this post!  
Unread 06-11-2003, 08:00 PM
Jeff Waegelin's Avatar
Jeff Waegelin Jeff Waegelin is offline
El Jefe de 148
AKA: Midwest Refugee
FRC #0148 (Robowranglers)
Team Role: Engineer
 
Join Date: Aug 2001
Rookie Year: 2001
Location: Greenville, TX
Posts: 3,132
Jeff Waegelin has a reputation beyond reputeJeff Waegelin has a reputation beyond reputeJeff Waegelin has a reputation beyond reputeJeff Waegelin has a reputation beyond reputeJeff Waegelin has a reputation beyond reputeJeff Waegelin has a reputation beyond reputeJeff Waegelin has a reputation beyond reputeJeff Waegelin has a reputation beyond reputeJeff Waegelin has a reputation beyond reputeJeff Waegelin has a reputation beyond reputeJeff Waegelin has a reputation beyond repute
Requirements- What Data to Collect?

Well, this is the first of what will be a series of threads to determine the necessary requirements for a viable FSN system. Now that I've thrown out a bunch of legal-sounding garbage, let me say this in plain English. We want to know what you want FSN to do. These "Requirements" threads will attempt to figure that out.

The topic I chose to start with is a fairly big one: data collection. My question to the FIRST scouting community is this: what data should we collect, and how should we gather it? How can we best determine what data fields should be collected and submitted? Who will choose what is the "official standard," so to speak? If you have anything to say regarding the actual scouting data that should be collected, this is the place to do it!

Note: This is not about methods of collection- just the data itself. See my second question for collection methods.
__________________
Jeff Waegelin
Mechanical Engineer, Innovation First Labs
Lead Engineer, Team 148 - The Robowranglers

Last edited by Jeff Waegelin : 06-11-2003 at 08:06 PM.
  #2   Spotlight this post!  
Unread 06-11-2003, 08:37 PM
SarahB's Avatar
SarahB SarahB is offline
Registered User
no team
Team Role: Engineer
 
Join Date: Feb 2003
Rookie Year: 2002
Location: Syracuse, NY
Posts: 231
SarahB has a spectacular aura aboutSarahB has a spectacular aura aboutSarahB has a spectacular aura about
Send a message via AIM to SarahB
A lot of this depends on what the game is next year. There are lots of game specific things such as stacking ability/speed for this year. These fields can be discussed here during the build season, then finally decided by a popular vote or some such thing.

One thing you can guarantee will be necessary is drive train. For preliminary scouting, fields for motors used, gear ratio, and shifting ability would be perfect. In match evaluation is a little harder to do objectively. A time to the top of the ramp(or something equivalent to fit with next year’s game) could be used for speed, but then you’d have to keep in mind that some teams have their autonomous run at a slower speed. A subjective rating(1-5) could be used for both, but that poses problems with different people having different standards/ratings. Earlier Erin suggested using a system where the mode is displayed on the main page and the percentages are shown on another page. This could help eliminate the subjectivity problems. Autonomous mode is also going to play some role next year. I would suggest two different evaluations for that. One would be a simple description of what their program does. A list could be made of the most common things(Up the ramp, attack human player bins, etc for this year) along with a miscellaneous option with a field to enter a description. The other would be another 1-5 rating of the quality. How fast is it, how effective is it, stuff like that.
__________________
FRC:
Team 30 - FURY (Student, 2002-2004)
Team 229 - Division by Zero (Mentor, 2005-2007)
FLL:
Robo-Monkeys (Founder/Mentor, 2003-2006)

Currently working as a Radar Systems Engineer at Lockheed Martin in Liverpool, NY and pursuing my MSEE from Clarkson, while easing my way back into FIRST
  #3   Spotlight this post!  
Unread 06-12-2003, 12:25 AM
Mike Soukup's Avatar
Mike Soukup Mike Soukup is offline
Software guy
FRC #0111 (Wildstang)
Team Role: Engineer
 
Join Date: May 2001
Rookie Year: 1996
Location: Schaumburg, IL
Posts: 797
Mike Soukup has a reputation beyond reputeMike Soukup has a reputation beyond reputeMike Soukup has a reputation beyond reputeMike Soukup has a reputation beyond reputeMike Soukup has a reputation beyond reputeMike Soukup has a reputation beyond reputeMike Soukup has a reputation beyond reputeMike Soukup has a reputation beyond reputeMike Soukup has a reputation beyond reputeMike Soukup has a reputation beyond reputeMike Soukup has a reputation beyond repute
I'll try to help you out in generalities and tell you what worked for me and what didn't...

When I did WASH in 2001 & 2002 I tried my hardest to keep the fields numerical or time based so the summary sheets could give meaningful information such as average, min, max, etc. I recommend that you do the same. I also found it hard to impose a 1-5 rating for a robot's performance because unfortunately a robot of quality 4 to me is not a robot of quality 4 to someone else. But I did it anyway and tried to train the scouts. I also found that it helps to assign meanings to the 1-5 ratings, such as 1 for traction means the robot could be pushed sideways, and a 5 means there was no slippage. Stuff like that.

I also found that for match scouting you should keep it as simple & concise as possible. I made the mistake of having too many fields and it just confused the scouts. They didn't have enough time and they had to rush to finish filling out the forms.

Finally and most importantly, design your system so you can easily and quickly modify the fields you gather. The game is never played out like we first invision it so the scouting system must be able to adapt. It should take no more than 2-3 days to add or remove a field from the DB, web interface, and input programs because that's all the time you'll have between regionals to work on it. Trust me, I've been there

Keep up the good work everyone.

Mike
  #4   Spotlight this post!  
Unread 06-12-2003, 12:53 AM
Brandon Martus's Avatar Unsung FIRST Hero
Brandon Martus Brandon Martus is offline
busy.
AKA: B. Slash Kamen
no team
 
Join Date: May 2001
Rookie Year: 1998
Location: Nevada, TX USA
Posts: 5,272
Brandon Martus has a reputation beyond reputeBrandon Martus has a reputation beyond reputeBrandon Martus has a reputation beyond reputeBrandon Martus has a reputation beyond reputeBrandon Martus has a reputation beyond reputeBrandon Martus has a reputation beyond reputeBrandon Martus has a reputation beyond reputeBrandon Martus has a reputation beyond reputeBrandon Martus has a reputation beyond reputeBrandon Martus has a reputation beyond reputeBrandon Martus has a reputation beyond repute
Send a message via ICQ to Brandon Martus Send a message via AIM to Brandon Martus Send a message via Yahoo to Brandon Martus
I'll expand on that just a bit..

Flexibility is the key word you've got to run through your head a few times while designing the base system. You have to think about not only how the game may change from Jan->April, but how the game could change from this year -> 2 years from now. If you are planning on keeping any historical data, you will want to be able to either expand/change the game requirements without having to start new databases or totally re-design it every year and then convert tons of old data -> new data.

Just something to think about.
__________________
Brandon Martus
e-mail
  #5   Spotlight this post!  
Unread 06-12-2003, 11:42 AM
Lindsey Lindsey is offline
Registered User
FRC #0862 (Lightning Robotics)
Team Role: Mentor
 
Join Date: Mar 2003
Rookie Year: 2002
Location: Northville, MI
Posts: 42
Lindsey will become famous soon enough
As far as what data to collect it rely depends on next year's game but there's standard stuff:

Pitt Scouting:
- Team #
- Type of drive (2 whl, 4 whl, tank, crab, ...)
- Drive motors used
- info about transmission (have/don't, brake?, how many gears, ...)
- # of modules/attachments
- capabilities (ex. stack)

Match Scouting
- Team #
- Partner
- Opponents
- Match score
- QP score
- Other alliance's match score
- Other alliance's QP score
- If they won
- Speed
- Traction
- Evaluation of capabilities (ex. how well they stack)
If they have auto. next year
- What it does
- How well it works

With the match and QP scores you could just fill in the match scores and whether or not you won and let the QP score be a calculated field. Or you could just enter Alli he info.

I agree that evaluating capabilities gets very subjective. As said before just using # 1 to 5 doesn't work because one person's 5 may be another's 3. This year my team got around the number thing we had a list of choices N/A, not functioning, marginal, successful, and amazing. This worked pretty good but you couldn't average or total scores. So I've been thinking why not use percentages! They're universal a 90% in Michigan is the same as a 90% in California. Also, everyone knows what the numbers mean: 60's - below average/not working well, 70's - average/works, 80's above average/works great ... With percentages you need to explain the everyone knows what they mean, they're great!

Just my latest crazy idea
Lindsey
  #6   Spotlight this post!  
Unread 06-12-2003, 01:20 PM
Beth Sweet's Avatar
Beth Sweet Beth Sweet is offline
is getting lost in her new home
FRC #0116 (Epsilon Delta), #1504 (alum), #67 (alum)
 
Join Date: Mar 2003
Rookie Year: 2003
Location: Reston, VA
Posts: 1,938
Beth Sweet has a reputation beyond reputeBeth Sweet has a reputation beyond reputeBeth Sweet has a reputation beyond reputeBeth Sweet has a reputation beyond reputeBeth Sweet has a reputation beyond reputeBeth Sweet has a reputation beyond reputeBeth Sweet has a reputation beyond reputeBeth Sweet has a reputation beyond reputeBeth Sweet has a reputation beyond reputeBeth Sweet has a reputation beyond reputeBeth Sweet has a reputation beyond repute
Hey there,
I just registered on the FSN, and there were a couple of things that I think for sure that you should add. For 1, you definately need to know what team each of the members is from. Also, it would help to have a page w/ a list of all of the teams, and possibly the team websites attatched. Probably a good idea to have a link for each team, and to get as much info about them on there as possible. History, past awards, pics, sponsors??? and basic info would probably be good things to include too. If I knew anything about how to work a website, I'd offer to help, but I'm still trying to figure out how to get my picture up on this one!
  #7   Spotlight this post!  
Unread 06-12-2003, 07:18 PM
John JediMaster's Avatar
John JediMaster John JediMaster is offline
Registered User
#0019 (Big Red)
 
Join Date: Apr 2003
Location: Greenwich
Posts: 67
John JediMaster is an unknown quantity at this point
Send a message via AIM to John JediMaster
try to include what their stat would be, this wouldn't be possible untill comp season though. also if possible give some basic names for their autonomous mode, it's gonna be back next year. and then put their am into one of those categories.
  #8   Spotlight this post!  
Unread 06-12-2003, 10:58 PM
AJ Quick's Avatar
AJ Quick AJ Quick is offline
Your Attention Please
FRC #0006 (CogSquad)
Team Role: Alumni
 
Join Date: Oct 2002
Rookie Year: 2001
Location: Minnesota
Posts: 464
AJ Quick has a spectacular aura aboutAJ Quick has a spectacular aura about
You need to make this as simple as possilbe. The less number of forms one needs to fill out the better... use some very descriptive fields and leave all reoccuring fields seperate (Ie Robot Name). Hopefully with a system all information will be filled out weeks before a competition and will be available for all teams.
__________________
-AJ Quick (Alum on Team 6)
http://www.HHSRobotics.com/

Place your attention entirely on my voice and try to ignore all the others. It is important to be able to focus on a single source of information while blocking out all the other things competing for your attention.
  #9   Spotlight this post!  
Unread 06-13-2003, 05:49 PM
Mike Soukup's Avatar
Mike Soukup Mike Soukup is offline
Software guy
FRC #0111 (Wildstang)
Team Role: Engineer
 
Join Date: May 2001
Rookie Year: 1996
Location: Schaumburg, IL
Posts: 797
Mike Soukup has a reputation beyond reputeMike Soukup has a reputation beyond reputeMike Soukup has a reputation beyond reputeMike Soukup has a reputation beyond reputeMike Soukup has a reputation beyond reputeMike Soukup has a reputation beyond reputeMike Soukup has a reputation beyond reputeMike Soukup has a reputation beyond reputeMike Soukup has a reputation beyond reputeMike Soukup has a reputation beyond reputeMike Soukup has a reputation beyond repute
Maybe this post belongs in the other thread, so move it if it more appropriate there.

Make sure you properly seperate the data you're collecting (and design your database correctly) so that you collect data where & when it makes sense. Data that never changes - team number, team name, etc - should be gathered during the build season. Non-subjective robot data - 2/4wd, speed, stacker/ball collector, etc - should be filled in by teams whenever they feel comfortable revealing that info and by pit scouts. Subjective data - how well the robot collects/scores balls, moves goals, stacks bins, protects stacks, balances the goals, etc - should be collected during matches. Do not trust teams to tell you how well they perform. They either overstate their abilities or flat out lie. Match scoring data should not be collected by the scouts. Either data-mine from FIRST or (if you can't do it at comps) have a dedicated person do it.
  #10   Spotlight this post!  
Unread 06-20-2003, 05:38 PM
Woodie Flowers Award
Ken Patton Ken Patton is offline
purple
FRC #0051 (Wings of Fire)
Team Role: Engineer
 
Join Date: Jul 2001
Rookie Year: 1997
Location: Pontiac, MI
Posts: 335
Ken Patton has a reputation beyond reputeKen Patton has a reputation beyond reputeKen Patton has a reputation beyond reputeKen Patton has a reputation beyond reputeKen Patton has a reputation beyond reputeKen Patton has a reputation beyond reputeKen Patton has a reputation beyond reputeKen Patton has a reputation beyond reputeKen Patton has a reputation beyond reputeKen Patton has a reputation beyond reputeKen Patton has a reputation beyond repute
categories

Since this is an area of great interest to me I would like to put in my 2 cents.

First of all, the comments that Mike Soukup made re:scouting are worth rereading - he is making good points from an experience base that few of us have. One key point is that ease-of-recording, simplicity, clarity - whatever you want to call it - is a huge factor in getting high-quality data from multiple sources.


A couple of people have suggested categories of scouting data - I agree, and suggest the following four categories:

1. ROBOT FEATURES - descriptions of the hardware/software based on observations prior to or during events; can be self-submitted or objectively observed by others
Example: # of drive motors

2. ROBOT RATINGS - subjective ratings made based on observations of actual robot performance during practice or matches; require consistent observation and the judgement of dedicated scouts
Example: Pushing Ability (1=poor, 5=great)

3. ROBOT RESULTS - specific outcomes that must be observed during practice or matches; require consistent observation by dedicated scouts
Example: First to the top (yes/no); Defend Top (yes/no); Tallest Stack Made (#bins)

4. MATCH INFO - basic match data taken from FIRST match listings and match results
Example: Match Points (#points)

I think all the gathered data will fall into one of these categories. What do you think?

Ken
  #11   Spotlight this post!  
Unread 06-21-2003, 12:54 AM
Andrew Andrew is offline
Registered User
#0356
 
Join Date: May 2002
Location: Little Rock, AR
Posts: 393
Andrew is a name known to allAndrew is a name known to allAndrew is a name known to allAndrew is a name known to allAndrew is a name known to allAndrew is a name known to all
We actually video-taped all of practice day and all of day one and watched tapes back at the hotel after each event. We didn't really attempt to make "during the match" scouting, since there is too much going on.

Hint. Hint. FSN might want to get a camera to each regional and do the info dump off-line. This is an unbelievably enormous task. BUt, I suspect that teams would contribute film for the good of the cause.

We believe in quantitative stuff.

Empirical speed of robot is more important that whether the team has a transmission or a particular gear ratio. For last year, we would care about time to top of the ramp during autonomous, time over a short distance (eg 3 meters), time over a long distance (eg 10 meters).

Whatever the game, I think time to the "first objective" is an important stat.
2000: time to far side of field
2001: time to the near side of the bridge
2002: time to the goals in the center
2003: time to the top of the ramp

Rather than having a power stat, you would like to see how teams actually fare against each other in competition. Ie Team XXX pushed Team YYY in match ZZZ. Later, Team YYY pushed Team WWW in match VVV. You can start to form a power class based on who pushed whom.

Robot weight is a really important stat and this should be obtained at the inspection station. I believe FIRST actually records this stat.

I agree with Ken's Kategories.

On match info, points scored, points allowed, QPs, win/loss.

We also try to accumulate average rank of alliance partners and average rank of opponents (computed at the end of the Qualifying).

Now, if FIRST throws a 4v0 plan against us again, most of the match data would have to be refigured.

Additional Game Specific Match info would be maybe necessary (even though hard to collect):
2000: number of balls robot put into trough
2001: big balls picked up and placed (even if subsequently knocked off), little balls scored in the goal, bridge balanced
2002: balls placed in goal, goals placed in zones
2003: bins stacked (ha. ha.), part of wall knocked over (20%, 50%, 100%), bins descored (10%, ...), on ramp

This last would require film.

An additional Kategory: Human Player Performance.
Closed Thread


Thread Tools
Display Modes Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Requirements- How to collect data? Jeff Waegelin Scouting 8 06-15-2003 12:57 PM
Competition Scouting pauluffel Scouting 31 01-09-2003 12:34 AM
Robot Scouting Database archiver General Forum 10 06-24-2002 03:00 AM
Limiting PWM in pbasic archiver General Forum 4 06-24-2002 12:38 AM
Distribution of data soap108 Scouting 4 12-24-2001 09:23 AM


All times are GMT -5. The time now is 09:52 PM.

The Chief Delphi Forums are sponsored by Innovation First International, Inc.


Powered by vBulletin® Version 3.6.4
Copyright ©2000 - 2018, Jelsoft Enterprises Ltd.
Copyright © Chief Delphi