Chief Delphi

Chief Delphi (http://www.chiefdelphi.com/forums/index.php)
-   Scouting (http://www.chiefdelphi.com/forums/forumdisplay.php?f=36)
-   -   2017 Proposed Scouting Methods (http://www.chiefdelphi.com/forums/showthread.php?t=153989)

Matt Flickinger 20-01-2017 19:39

2017 Proposed Scouting Methods
 
Hello,
I am a lead scouting and strategy member for Truck Town Thunder, and a dilemma we've come up with is that it will be nearly impossible to count the number of balls going into the low goal, and very hard to count the number of balls going into the high goal when the shooters are very fast. I am posting this thread to ask what other methods other people have come up with to get past this problem, or to possibly brainstorm ideas.
Sincerely, MF.

6101 Robert 20-01-2017 19:56

Re: 2017 Proposed Scouting Methods
 
What if you have your scouts estimate the average capacity of the team's hopper when they unload it (to the nearest 10^1) and count the number of times a teams unloads their hopper.

That wouldn't be accurate enough of a system to select alliance partners based simply off of an algorithm, but you would be able to graph (or otherwise display) how teams stacked up against each other in terms of fuel. To increase the accuracy of your estimation, you could ask them directly what their maximum capacity is.

JR0405 20-01-2017 19:59

Re: 2017 Proposed Scouting Methods
 
As other people in other threads have said, this years game will require a lot more qualitative and opinion based scouting then past years. So instead of this team made 7 out of 10 shots your scouting sheet or app would have different groups of percentages.

Jarren Harkema 20-01-2017 20:16

Re: 2017 Proposed Scouting Methods
 
High speed camera footage. The only problem is finding someone willing to sit in a corner the entire tournament watching 1/2 or 1/4 speed footage and count balls flying in the air. I'm not sure if there are enough sugary treats in the world to bribe someone into watching that all day long though.

In all serious, some sort of estimation is going to be needed. I'm curious to see what data the FMS is going to record. If it distinguishes between high and low boiler fuel, you may be able to run an OPR-like algorithm against those values, and then compare to your scouters' estimated values. It's not going perfect, but it provide that balancing force if two different scouters estimate differently. This won't account for accuracy obviously. Arguably accuracy matters less if the total number of balls scored on average by one team is better than another during the entire length of the match.

Billfred 20-01-2017 20:32

Re: 2017 Proposed Scouting Methods
 
Quote:

Originally Posted by Jarren Harkema (Post 1634218)
In all serious, some sort of estimation is going to be needed. I'm curious to see what data the FMS is going to record. If it distinguishes between high and low boiler fuel, you may be able to run an OPR-like algorithm against those values, and then compare to your scouters' estimated values. It's not going perfect, but it provide that balancing force if two different scouters estimate differently. This won't account for accuracy obviously. Arguably accuracy matters less if the total number of balls scored on average by one team is better than another during the entire length of the match.

If FIRST releases the breakdown of data in their system (and what they made available for FIRST Stronghold makes me think they could), I might reallocate a 6-scout crew this way:

2 on Boilers: Divide the high and low boiler action across members of the alliance. (This is an estimate, but it changes the game from "count balls" to "who's doing a lot over there?" and makes the numbers equally screwy for the alliance since one person is grading all three.) Take these percentages and allocate balls to each robot accordingly.
2 on Airships: Who delivers gears? Who boards? Do we see overflow chute action (which is a mark against the other alliance)? Do we see the lift handles getting knocked out of the ports (which is a huge mark against the alliance onboard)?
1 on Hoppers: Who's getting dumps? When? How much is going in the robot? (I don't care whether they're scoring it here--I have people watching the boilers for that.)
1 on Neutral Zone: Who's collecting a lot? Who's dishing defense, or avoiding it well?

(I wonder how long it'll be until some team starts farming this data coding out as Mechanical Turk tasks using someone's unlimited data plan...)

Edxu 20-01-2017 20:34

Re: 2017 Proposed Scouting Methods
 
You could estimate their balls scored with variance of 3 for high and 9 for low by simply counting the score and seeing if that value is reasonable or not by inspection.

Since gear values go up in intervals of 40, any changes in the ones column must be from scored balls, so if you can tell if the balls are scored high or low, then you should be okay.

However, this system breaks when more than 1 robot is scoring at a time, so good luck with that.

Joy4201 20-01-2017 20:45

Re: 2017 Proposed Scouting Methods
 
This question (with lots of suggestions on how to solve the problem) are also here: https://www.chiefdelphi.com/forums/s...light=low+goal

New Lightning 20-01-2017 21:40

Re: 2017 Proposed Scouting Methods
 
What I think could be a potential solution would be to still try and count the number of balls scored per emptying of a teams hopper and then compare that number to how many point were just scored.

Hopefully the FMS would do something similar to 2014 with number of assist with the three green dots when there is a partial kPa scored in the boiler, have three green dots above the total score for the high goal and 9 dots below to represent the low goal. Your scouts would have to be attentive to what the score is prior to the scoring and what the score is now post ball scoring. That way the can do the math to figure out how much was actually scored by the robot and then multiply by either 3 or 9 depending on how many points were scored.

SPang 20-01-2017 21:42

Re: 2017 Proposed Scouting Methods
 
You can also simply have your pit scouters ask teams about their hopper capacity. There's going to be the usual dissonance between reality and what teams believe they're capable of, but it helps with making estimates more accurate to the robot's actual scoring ability. You can probably amend this by subtracting a certain percentage based on ball packing or how full they actually fill their hopper while cycling in matches.

It's much less effort to make educated approximations rather than spending a disproportionate amount of time on trying to be slightly more accurate.

Jarren Harkema 21-01-2017 00:22

Re: 2017 Proposed Scouting Methods
 
Quote:

Originally Posted by Billfred (Post 1634224)
If FIRST releases the breakdown of data in their system (and what they made available for FIRST Stronghold makes me think they could), I might reallocate a 6-scout crew this way:

2 on Boilers: Divide the high and low boiler action across members of the alliance. (This is an estimate, but it changes the game from "count balls" to "who's doing a lot over there?" and makes the numbers equally screwy for the alliance since one person is grading all three.) Take these percentages and allocate balls to each robot accordingly.
2 on Airships: Who delivers gears? Who boards? Do we see overflow chute action (which is a mark against the other alliance)? Do we see the lift handles getting knocked out of the ports (which is a huge mark against the alliance onboard)?
1 on Hoppers: Who's getting dumps? When? How much is going in the robot? (I don't care whether they're scoring it here--I have people watching the boilers for that.)
1 on Neutral Zone: Who's collecting a lot? Who's dishing defense, or avoiding it well?

(I wonder how long it'll be until some team starts farming this data coding out as Mechanical Turk tasks using someone's unlimited data plan...)

This is an interesting idea. Kind of reminds me how referees observe matches. Don't they typically watch certain locations rather than specific robots? Theoretically the data collected would be much more consistent between matches. That's my biggest fear of an estimation system; my 50% guess is your 60% guess.

I will bring this up to our scouting team and see if there is anything they can take from it.

Quote:

Originally Posted by SPang (Post 1634250)
You can also simply have your pit scouters ask teams about their hopper capacity. There's going to be the usual dissonance between reality and what teams believe they're capable of, but it helps with making estimates more accurate to the robot's actual scoring ability. You can probably amend this by subtracting a certain percentage based on ball packing or how full they actually fill their hopper while cycling in matches.

It's much less effort to make educated approximations rather than spending a disproportionate amount of time on trying to be slightly more accurate.

I'm just waiting to see a student walking around in the pits with a trash bag slung over the shoulder full of fuel. He or she just walks up, dumps the contents of the bag into a robot (asking first of course), and then picks them out while counting. He or she then makes a note, thanks the pit crew, and hurries on to the next team.

Skyehawk 26-01-2017 11:53

Re: 2017 Proposed Scouting Methods
 
Quote:

Originally Posted by Jarren Harkema (Post 1634321)
I'm just waiting to see a student walking around in the pits with a trash bag slung over the shoulder full of fuel. He or she just walks up, dumps the contents of the bag into a robot (asking first of course), and then picks them out while counting. He or she then makes a note, thanks the pit crew, and hurries on to the next team.

I'm. So. Tempted.

In reality though, I was going to just measure the inside of their hopper and use the loose random packing density of spheres ~60-61% as my number. If a team is willing to verify with an actual random fuel dump I will give them points for honesty. Honesty points may move them on our pick lists.

MARS_James 26-01-2017 12:03

Re: 2017 Proposed Scouting Methods
 
Quote:

Originally Posted by Jarren Harkema (Post 1634321)
I'm just waiting to see a student walking around in the pits with a trash bag slung over the shoulder full of fuel. He or she just walks up, dumps the contents of the bag into a robot (asking first of course), and then picks them out while counting. He or she then makes a note, thanks the pit crew, and hurries on to the next team.

That seems counter productive if people design large hoppers, why not just have a bag of 50 or so, and count the ones that don't fit?

Trevor1523 26-01-2017 12:06

Re: 2017 Proposed Scouting Methods
 
This was spoken about last night at our build space. Yes, it is nearly impossible to count the amount of fuel going into the boiler. At the regional level it isn't really relevant.

Something I would focus more on is how accurate teams are at getting fuel into the boiler. It doesn't really count if one team can shoot 100 fuel at the high goal and only have 10-20 go in because their shooter isn't accurate. But if there's a team that can shoot 40-50 fuel at one time and only miss a few? That's a lot more valuable and a lot less likely to waste time during the match.

In summation; just look for how efficient/accurate teams shooter's are.

Skyehawk 26-01-2017 12:10

Re: 2017 Proposed Scouting Methods
 
Quote:

Originally Posted by Trevor1523 (Post 1636636)
This was spoken about last night at our build space. Yes, it is nearly impossible to count the amount of fuel going into the boiler. At the regional level it isn't really relevant.

Something I would focus more on is how accurate teams are at getting fuel into the boiler. It doesn't really count if one team can shoot 100 fuel at the high goal and only have 10-20 go in because their shooter isn't accurate. But if there's a team that can shoot 40-50 fuel at one time and only miss a few? That's a lot more valuable and a lot less likely to waste time during the match.

In summation; just look for how efficient teams shooter's are.

To take your example to the extreme:
If I had a fuel cycle in which I shot 9 balls (my entire hopper) and made 8/9 of them in 10 seconds I would be ranked higher than a team the shoots 75 balls in 10 seconds and makes 50 of them? You need some way to bring your ratio (make %) onto a level basis for comparison.

Just some thoughts.

JesseK 26-01-2017 12:25

Re: 2017 Proposed Scouting Methods
 
Quote:

Originally Posted by Billfred (Post 1634224)
... "who's doing a lot over there?" ...

Presuming FIRST FMS releases data on BOILER points, I think Billfred hit the nail on the head with the fuzzy logic: do they spend a lot of time at the boiler, or not?

Record a fuzzy group number for each load of balls taken to the boiler - 1=Less than 10-count, 3=more than 30-count, or 2=in between. Put a Gold Star if the team seemed to focus on fuel that match.

Take all gold star bots, divide amount of boiler points per match by the average grouping. Voila! Imperfect, but possibly-good-enough way to determine the best fuel bots at a competition. It won't order them bot-for-bot, but at most events bots rated in the middle-of-the-pack for fuel will likely be used in elims roles that include, but do not revolve around, fuel.

Trevor1523 26-01-2017 12:32

Re: 2017 Proposed Scouting Methods
 
Quote:

Originally Posted by Skyehawk (Post 1636638)
To take your example to the extreme:
If I had a fuel cycle in which I shot 9 balls (my entire hopper) and made 8/9 of them in 10 seconds I would be ranked higher than a team the shoots 75 balls in 10 seconds and makes 50 of them? You need some way to bring your ratio (make %) onto a level basis for comparison.

Just some thoughts.

Well it wouldn't necessarily be that crazy. I think we both know that 75 balls in 10 seconds is quite optimistic. I would actually prefer the 50 over the 9, because it's more balls per second. But if it took a lot longer to score, and the robot misses a lot of shots then theoretically the robot is just sitting there and not scoring points. I would be looking for robots that are efficient. I think that makes sense.

Skyehawk 26-01-2017 13:18

Re: 2017 Proposed Scouting Methods
 
Quote:

Originally Posted by Trevor1523 (Post 1636648)
Well it wouldn't necessarily be that crazy. I think we both know that 75 balls in 10 seconds is quite optimistic. I would actually prefer the 50 over the 9, because it's more balls per second. But if it took a lot longer to score, and the robot misses a lot of shots then theoretically the robot is just sitting there and not scoring points. I would be looking for robots that are efficient. I think that makes sense.

My point is (and maybe you know this) you don't know the average amount of fuel shot information with the % method you are proposing. I will be releasing what data we will be collecting for this year on Feb 7th. While shots made % is a category we are tracking it has two other data-points to make the % a usable (and therefor worthwhile) data-point to track.

Hypothetical data (assuming constant cycle time):
Team # | ~High % | ~Load Size
001 33 20
002 50 50
003 100 10

If you only have the first two columns (Team # and make%) how are you going to differentiate what teams score the most? You will need another way to bring the make% into focus. If teams always used the full extent of their hopper you could use the size of their hopper, but there will not always bee full hoppers. I recommend including a 1/3 full, 2/3 full, and 3/3 full option just to get an easy average of their match load size. Cross reference this with how large their hopper is (from pit scouting or careful observation) and BOOM you can calculate ~fuel pts./match.


All times are GMT -5. The time now is 14:34.

Powered by vBulletin® Version 3.6.4
Copyright ©2000 - 2017, Jelsoft Enterprises Ltd.
Copyright © Chief Delphi