[2017] Accuracy of scouting boiler goals...
 

After analyzing how scouts would potentially go about tracking a robot's fuel scoring ability, I came to the conclusion that there is virtually no way scouts will be able to make an accurate count of how many "fuel" balls are scored in the lower goal of the boiler. A cursory examination of the game leads me to believe that many robots will be scoring at this goal in bulk (think a "dump truck" robot design) that could be putting anything from ten to a hundred balls into the goal. There is no way that a scout will be able to accurately count each fuel ball that goes into the lower goal. This inexactness is a potentially scary problem that should be recognized by the greater scouting community.

How do you guys think we should adjust our scouting procedures to adjust for this potential inaccuracy?

Re: [2017] Accuracy of scouting boiler goals...
 

One thing that was suggested on my team was to count cycles. How fast can a robot grab a good number of balls and dump? We also thought that we should make an estimate of the balls they have at each dump (maybe just low, medium, high).

Re: [2017] Accuracy of scouting boiler goals...
 

If you filled the entire volume of your robot with spheres, you'd only be able to fit 50 spheres in your robot. Proof:

Maximum volume of robot, M=34,560 inches cubed.
Volume of sphere, S=4/3*PI*5^3=523.6 inches cubed
Optimum packing order of spheres, P=0.74
Maximum amount of balls in robot: M*P/S=49. Q.E.D

Re: [2017] Accuracy of scouting boiler goals...
 

Good idea! In addition, scouts can estimate the accuracy too, and multiply that by the maximum capacity that team claims they can hold to get another reading of how many balls were scored per cycle.

I would imagine that high goal is easier to estimate than low goal though.

Re: [2017] Accuracy of scouting boiler goals...
 

Not quite. The diameter of the sphere is 5", so the radius is 2.5". You're off by a factor of 8.

Edit: Also, the balls are slightly squishy. In addition, you can pile a hill of balls higher than your robot max height and hold them all.

Re: [2017] Accuracy of scouting boiler goals...
 

I have been thinking about this as well. My scouting application (https://www.frcscout.resnick-tech.com), which is open for every team to utilize, just has a button for a high or low goal. I was thinking about how to do bulk scoring and I think I will just have a drop down with increments and allow users to select the number to be submitted. Please feel free to take a look and let me know.

Re: [2017] Accuracy of scouting boiler goals...
 

Last year FMS tracked the high and low goals in teleop separately. I don't know if that will happen again this year or if it will happen in auto too, but TBA may be helpful in this regard (you would have to note if it seems if only one or if two or more robots are scoring in the low efficiency goal, but it may be better than nothing).

Re: [2017] Accuracy of scouting boiler goals...
 

For purposes of the kPa calculation, the FMS will have to keep track of Auto/TeleOp separately since Auto fuel counts for 3 times as much pressure. Also keep in mind that the processing of the fuel for Auto stops when Auto stops and anything left in the hopper is now TeleOp fuel. At the end of the match, the same holds true when at t=0, processing stops regardless of what was still in the hopper at the time. There is no "the ball was in the air so it counts" with this game.

Re: [2017] Accuracy of scouting boiler goals...
 

They will definitely keep track of the number of balls (fuel). The question is whether scouts will be able to determine in the moment the amount. Also the FMS does not determine which robot did what, so there is no way to determine who contributed what number of balls based on the FMS / Blue Alliance data.

Re: [2017] Accuracy of scouting boiler goals...
 

We'll probably make high ball scoring fall into 3 catagories:
A) Amazing!
B) Pretty good.
C) Meh.

Re: [2017] Accuracy of scouting boiler goals...
 

We're going to hopefully be better than 'Meh'! :)

Steve

Re: [2017] Accuracy of scouting boiler goals...
 

You're right. Redo:

Maximum volume of robot, M=34,560 inches cubed.
Volume of sphere, S=4/3*PI*2.5^3=65.5 inches cubed
Optimum packing order of spheres, P=0.74
Maximum amount of balls in robot: M*P/S=391.

I revoke my skepticism about 100 balls being dumped into a goal.

Re: [2017] Accuracy of scouting boiler goals...
 

Don't forget that that volume includes the bumpers... ~3.25 inches from each side of the robot that the frame perimeter must be held within. So maximally, robots themselves can actually only be 23718 cubic inches (and thats in the short orientation, its actually less in the tall orientation, at 21573 cubic inches).

Keep in mind that the random packing density is maximally 64% (optimal planned packing is higher, but thats not going to happen during a match). So we take the number of fuel that an empty space of the size that the robot could fit in to be only 231. Now consider that the robot needs electronics, a frame, wheels, and some mechanism to actually dump the balls, and I think you'd be lucky to see 100, maybe 150 ball dumps, if that's all the robot does.

Now for the discussion of scouting, we are considering the following for our electronic scouting app (that we plan to release before competitions start):

- go around before matches start and ask teams what the maximum number of fuel their robot can hold is, save that data in our app
- during match scouting, have a slider to estimate (to nearest 5 for low goal) the number of fuel scored by a robot, with a marker / upper limit just above the maximum capacity that the team reported their robot being able to score. Gives our scouters a way of visually going - "their robot looks about half full, so must be half of the number they reported", etc.

Re: [2017] Accuracy of scouting boiler goals...
 

I've talked about this with a few other members of my team since I'm the scout guy on our team. I'm sort of at a loss. We've had "super scouters" before. It'd be 1 per alliance along with 1 per robot. That means we'll need 8 scouts a match which might be a problem for us. But I've thought about designating super scouters where their whole job is to count how many balls each team on that alliance scores in the goals. It will be really hard to count low goals but high should be fairly easy. Good thing I have 6 weeks to figure this out... :(

Re: [2017] Accuracy of scouting boiler goals...
 

Calculate teams' component OPR for boiler points if you can. If that isn't doable, just count cycles and guesstimate "less than 30 balls" or "more than 30 balls," or something like that.

Re: [2017] Accuracy of scouting boiler goals...
 

We're playing with unreasonable numbers either way, but I feel like I should point out that your fuel tank can indeed extend over your bumpers so long as it starts within your frame perimeter. The only mandatory volume loss is the volume of the bumpers themselves (but for that matter the battery and other required components).

In terms of scouting it, your first general pass won't need to be very specific. You might just write Low/Med/High on the forms and wait a few matches at your event to help your scouts get a feel of each and set standards based on your alliance selection needs. One of the most useful metrics will be how fast an alliance hits 40 kPa and how much (cycles and which goal) a team had to do with it. By the time specifics really matter (District and Half Champs), the community will have built up a whole jargon for handling this, and your scouts will have the benefit of experience. Too often I see teams get overwhelmed trying to collect more detail than they need or are ready for and miss the forest for the trees.

Re: [2017] Accuracy of scouting boiler goals...
 

So our team has come up with a solution to this by scouting the number of cycles of balls rather than how many. Now, we also have made sure the scouter knows to specify if it's a big medium or small load.If youd like to see our scouting sheets I will gladly share it with you or anyone else for that matter!

Re: [2017] Accuracy of scouting boiler goals...
 

I haven't thought this through in detail, but perhaps if the scouts monitor the pressure level before and following a dump, you can at least get an estimate. Of course, if somebody is feeding the high-efficiency boiler at the same time, this will cause significant uncertainty.

At a deeper level: It is difficult to see how the low efficiency boiler is going to matter in any significant way, unless a high boiler scorer is just a bit slower than needed to reach 40 kPa (and even then, it would be better to feed the high goal shooter somehow). (Assuming just the original 10 of 1 robot in auto), in order to get 40 kPa, you would need to put about 330 fuel into the low goal in 105-135 sec, which means about 3 per second if you want to climb the rope. As the low efficiency boiler only has a capacity of about 50-60 fuel, and processes it at a maximum rate of 5 per second, this means that you make about six deliveries of 60 fuel about 20 seconds apart from each other. Honestly, that's way more impressive than a robot that can deliver three deliveries of 50 fuel into the high efficiency boiler every 45 seconds.

Re: [2017] Accuracy of scouting boiler goals...
 

I think a lot of this conversation is ignoring a potentially important data source:

TBA API

The match endpoints in the API will have the 2017 game score breakdown. What exactly that will be is still TBD, but given past breakdowns, and the fact that high and low goals need to be separately tracked, the API should have an accurate scored count.

If you have 1 bot doing all the low goals, you can get an accurate count, same as if there's only 1 high shooter. Then have the scouts approximate the contributions when more than one bot is scoring in a goal.

Re: [2017] Accuracy of scouting boiler goals...
 

I don't think this is just a low goal issue, while it won't be impossible to count high goals scored it will certainly be difficult. In a high level match we're probably going to see upwards of 120 balls scored and that point I don't think it makes sense to ask scouts to count each ball scored.

The smarter move would be to ask scouts to count missed shots only instead of those scored. Also if you could get truthful pit scouting information about the ball capacity of a robot.

Re: [2017] Accuracy of scouting boiler goals...
 

I have talked earlier this year about qualitative scouting, this may be the perfect time for it. Keep in mind, at best scouting a team is an estimate. If you can approximate certain aspects of the game and back solve to get the variable (balls in goal) this shouldn't be too difficult. Keeping a count on team cycles, approximate balls scored (and where), and an approximate accuracy should lead to reasonably accurate data. The most important thing is to have a strong basis for comparison that is consistent and you have confidence in.

Re: [2017] Accuracy of scouting boiler goals...
 

Tracking the number of balls a team can hold in their robot, approximate % of that load they can get in the boiler, and the number of cycles they do in a match will help. I think it would be ridiculous to try and hand count every ball that goes into the boiler. In order for this to be effective you'd need to do pit scouting / good use of observation.

Just my thoughts.

Re: [2017] Accuracy of scouting boiler goals...
 

I couldn't agree more, this year scouting seems to be more about the confidence you have in your data than anything. Of course we will see how it all shakes out after the first week or two. My initial thoughts on fuel calculation followed the methodology of using approximation and back solving. And frankly there are only certain 'zones' of fuel scored that are important: if they score (when and where) and if they approach/exceed the 40kPa of pressure. If your window of certainty is narrow enough while still allowing for the inconsistencies of scouting then you should be in good shape.

Following the kPa changes is a good method of doing it, especially with robots that score in 'unique' goals, and it may very well be the most accurate method. However I see it has one fatal flaw in the big picture; the lights on the field denoting kPa are relatively difficult to see accurately and are themselves sort of an approximation. If one was to use the scoreboard to keep track of kPa you have a more accurate data source but lose valuable 'eyes-on-field' time.

It is my belief that approximating groups of 10 balls scored by a robot in a goal, the robot's approximate make percentage, and the number of cycles is the best way to solve this problem. Keeping track of 600 balls individually is a fools errand, it will not be fun for those doing it and kind of defeats the purpose of FIRST. This is a sport, it is supposed to be fun for those playing and spectating.

Re: [2017] Accuracy of scouting boiler goals...
 

SkyeHawk check your PMs :)

Re: [2017] Accuracy of scouting boiler goals...
 

For the high goals, my team had the idea of counting how many balls missed, rather than how many went in. This, combined with a measure of how many trips the bot can take, would make for an accurate representation of that robot's strength.

Re: [2017] Accuracy of scouting boiler goals...
 

We're currently breaking it down into scenarios, most of which cause ball numbers to be a non-issue:

1. Nobody does balls: easy.
2. One team does balls: use the API.
3. One team goes high, one low: Still, just use API.
4. Multiple teams go low, 1 or 0 high: Pit scout maximum carrying capacity, make best guess on percent of maximum dumped.
5. Multiple teams go high: This is the tough one. Try to get a good angle, and take your best guess.

Re: [2017] Accuracy of scouting boiler goals...
 

This is not a bad way to go about it, it is probably the most logical, least amount of work on the end user, and simplest to denote in a form of some sort. it does have quite a few moving parts. The most accurate/least-work way will be to take use the total amount of balls and try to solve the overconstrained system, similar to OPR, but that is never truly accurate as well.
One word of caution, the API sometimes take a few minutes for the match to update, this could cause an annoying backlog of paper slips, complicated implementations utilizing a network link, or editing of data within an app. I would spring for clear communication among the scouting team and use the values that appear on the official score at the end of the match. This always happens prior to the next match and you solve most of the negative points of the API at the cost of another user input.

I am thinking rough multiples of 5 or 10 balls should provide data with a reasonable degree of certainty. the fewer buttons that have to be pushed or tally marks to be made the better. Eyes-on-field = more better.

Regards,
Skye Leake

Re: [2017] Accuracy of scouting boiler goals...
 

This. The number one mistake I see in scouting is teams collecting a lot of detail that they don't need. For scoring, we are going to try to keep a count of cycles with general categories for number of balls in each cycle. We are hoping to use the scoring data from each match to get the amount of fuel scored and divide that up among the teams on each alliance based on our observations. The fallback position if this proves to tough is to use a scale (probably 5 points from awesome to bad) and cycle time as a proxy.

In any event, collecting relevant data is a great thing to do scouting wise. But in my experience you also need to have scouts making subjective judgments and recording those. Things like "they get in their allies way a lot" or "the driver adapts well to changing conditions on the field" or even "wow that driver is amazing." Over the years this has served us well in finding good alliance partners and in picking good strategies against opponents. I recall one year when we were the first pick of a really good team who refused our suggestion for the second pick because "we don't ever pick teams with mecanum drive." The mecanum drive robot was picked next, and proved to be the decisive factor in our loss to the other alliance. To their credit, the team that picked us came over to our pit as we were packing up and said "We really should have listened to you."