Best way to estimate the best defense bot

[dellagd]

Im sprucing up our scouting system for our next event and would really like do work defense into it.

As of now, we have 6 scouts (+ a head scout but he/she does different things), and they all watch every match.

I have heard of DPR, and have a crude understanding that its basically the same calc as OPR but using the opponents score, but Ive heard this is inaccurate.

I could give each scout a slider of 1 - 5 in "How well does robot play defense", but this is purely objective.

How does your team do it? Is DPR actually useful?

[Hallry]

We don't use DPR (or OPR, for that matter), but our experienced scouts do rank defensive teams. During our scouting meeting the night before eliminations, we then use those rankings and look back at our match footage (with every match recorded), to see how those teams perform and how consistent they are.

[Grim Tuesday]

We have considered implementing the following into our scouting database and haven't but might get around to it for Championships:

For reference, the main goal of our scouting is to record exactly how many points a robot scores in a match. The advantage this has over OPR is that robots can not be 'carried' to a higher position than they deserve, it is literally exactly how many points they individually scored that match.

The concept for a true DPR is to create a marker in this system that notates if a robot plays defense and upon whom they played it. We would then compare the amount a robot 'should score' aka their score in matches without defense to the score in their match where defense was played. A number would then be assigned to the defending robot for how many points or percentage of points a defender shut down.

[MrJohnston]

We look at several things:
* What sort of drive train does the bot have?
* What sort of traction are on the wheels?
* How fast is the robot?
* How well does it turn?
* Has the robot done well in pushing matches?
* How smart is the drive team? (i.e. do they commit fouls?)

They key is to scout the pits just as well as the games. Often the best defensive bots play no defense during qualifying matches.

[themccannman]

The idea behind DPR makes sense, however when used in practice it's wildly inaccurate and I would recommend not using it. Our team's defensive ranking system so far has proven to be extremely successful so far. We have 4 purely subjective categories pertaining to a robots ability to play defense and their actual defensive performance, and 2 defense scouts (one per alliance) to rate each individual robot. At the end of qualifications we compile the data into a spreadsheet with predetermined weighting for each category and it spits out a couple of numbers that tell us exactly how good every robot is at doing the things we are interested in. The most important part of our system however is that you need experienced people who know what to look for in a robot to be able to adequately judge their performance otherwise your numbers are worthless.

Edit: the person above me basically covered most of the categories we use to judge robots. We use:
-Drive train speed
-Driver ability
-Height (< 30")
-Blocking performance

Only the last category is actually affected by robots playing defense, the rest are affected by their apparent ability to play defense if we need it.

[2789_B_Garcia]

themccannman & MrJohnston, I'm the lead mentor and drive team coach for a defense-oriented team. If you have time, I'd be interested in your opinions on some match footage to tell me what you think about our team! I agree that OPR and DPR aren't helpful or accurate, and I've also used a more qualitative system for making defense strategy decisions.

Out of curiosity, why do you think the best defensive robots don't play defense during quals?

[fox46]

Out of curiosity, why do you think the best defensive robots don't play defense during quals?

I'm sure we all can agree that the most important quality to have in a defensive machine is a strong drivetrain. However, in recent years FIRST has been steadily reducing the attractiveness of defensive strategies. In this year's game, a good defense doesn't score you any points. For this reason, most teams who build strong drivetrains do it not for defensive capabilities, but rather because if you can't get to where you need to be, you can't execute your gameplay.

Team 2013's robot uses a 6 CIM motor drive in a 6-wheel skid-steer configuration, rolling on custom aluminum rims wrapped with 2.5" wide roughtop conveyor belting. The machine weighs in at 117.3 lbs.

By all standards, a very defense-capable machine. However, the real reason for having a strong drive this year is that if we can't get to and from the feeder station and pyramid, then it doesn't matter how good the shooter and climber are.

During gameplay, the only times we have played defensively are if one or both of the other systems are down and we have no other way to contribute to the alliance, or if we deem defensive play necessary, such as if the other alliance has us out-gunned.

Hence a very defense-capable team which very rarely uses it.

[Chris is me]

You can't objectively do it. Don't even try.

The best you can do is a Defense option on your scouting sheet. "None" / "Ineffective" / "Somewhat Effective" / "Strong" / "Complete Shutdown" should be your scale (1-5 is too vague), and that isn't so much to quantify it as much as it is so you can know on Friday night which teams tried to play defense and *roughly* how good they are at it. Pit scout a little and determine which robots are 4 CIMs (minimum) and have traction drives. Also, be sure to scout Saturday to see who shows up to play excellent D.

All that said, more likely than not you'll find 1-3 excellent defenders at a regional, maybe a dozen (if you're lucky) average defenders, and then a bunch of teams limping around, trying to score or something and really just not doing much. You'll probably have to use other offensive features (10 point hang, any autonomous functionality) as your main decision maker. You are going to need smart people sitting there on Saturday morning watching matches and looking for diamonds in the rough. That's how you find them.

[EricH]

You guys are almost all overlooking one thing (I've seen one mention so far). And this one thing could lose you the event.

Penalties.

Does the potential defense robot in question (whichever robot it happens to be) rack up their opponents' points from fouls or their alliances' points from fouls? If you have a potential defender that is constantly getting penalties, particularly "stupid" penalties, then they should probably not be very high on your list. Back when I was scouting, penalty points were given a weighting of -1 in our metrics to deduct "score" from that robot.

Now, if the robot takes a smart penalty, then you also want to note that. It means that they know that a 20-point foul is better than letting their opponent potentially get 50 points, for example.

[MrJohnston]

I agree with fox46's answer...

The strong drivetrain and grippy wheels mean that the robot can push most anybody around. Certainly, this is a huge benefit offensively, allowing such tasks as forcing itself to a feeder station or to an ideal place to maek a shot. However, used defensively, it can make it very difficutl for a full-court shooter to get to its feederstation or a little shuttlebot to blitz back-and-forth between the pyramid and feeder.... Or, a really tall blocker can be attached mid-compeititon to it to deal with a full-court shooter as it would be nearly impossible to move.

Often teams with the where-with-all to build such a strong drivetrain, realize that many teams can't score. So, they need to have some scoring ability (a shooter, for instance). If the team is good enough to do this, their shooter will work well. So.. even though they are thinking "defense" they might must be the best offensive robot on the field in a qualification match.

[fox46]

You guys are almost all overlooking one thing (I've seen one mention so far). And this one thing could lose you the event.

Penalties.

All robots can incurr penalties- defensive or offensive. This should be considered for all teams you are scouting. It's far more likely to clip a climbing robot on the way to a feeder station than playing D with half a brain. Running back and forth accross the field you aren't focussing as much on the activities of other machines than if you were covering them playing D.

[dellagd]

You guys are almost all overlooking one thing (I've seen one mention so far). And this one thing could lose you the event.

Penalties.

Thats a main point of our system right now and we are sure to keep track of it.

"None" / "Ineffective" / "Somewhat Effective" / "Strong" / "Complete Shutdown"

All that said, more likely than not you'll find 1-3 excellent defenders at a regional, maybe a dozen (if you're lucky) average defenders, and then a bunch of teams limping around, trying to score or something and really just not doing much. You'll probably have to use other offensive features (10 point hang, any autonomous functionality) as your main decision maker.

I've tried the 1-5 scale of None, Ineffective, ect. but that got shot down by the scouts when we tried that. I'll try for it again. Hopefully working in some pit scouting into that will work out nicely.

Good to know DPR is no good.

Any other Ideas?

[fox46]

Case in point:

The strong drivetrain and grippy wheels mean that the robot can push most anybody around. Certainly, this is a huge benefit offensively, allowing such tasks as forcing itself to a feeder station or to an ideal place to maek a shot. However, used defensively, it can make it very difficutl for a full-court shooter to get to its feederstation or a little shuttlebot to blitz back-and-forth between the pyramid and feeder....

http://watchfirstnow.com/archive3.php?id=62936455

[themccannman]

Originally Posted by dellagd

I've tried the 1-5 scale of None, Ineffective, ect. but that got shot down by the scouts when we tried that. I'll try for it again. Hopefully working in some pit scouting into that will work out nicely.

Good to know DPR is no good.

Any other Ideas?

We do ours with a similar scale, 0 is a average robot, (-) points are if they perform poorly or are a detriment to their team and (+) points are if they perform above average. I would highly recommend a system like that or the 1-5 system you tried before. Tell the scouts to come up with a better system if they don't like it, otherwise just use that.

Originally Posted by EricH

You guys are almost all overlooking one thing (I've seen one mention so far). And this one thing could lose you the event.

Our "driver ability" category is supposed to cover this, if they are consistently causing penalties we will score them very poorly in that aspect. We've found that there tends to be a fairly close correlation between apparent driver skill level and average penalties incurred.

Originally Posted by 2789_B_Garcia

themccannman & MrJohnston, I'm the lead mentor and drive team coach for a defense-oriented team. If you have time, I'd be interested in your opinions on some match footage to tell me what you think about our team! I agree that OPR and DPR aren't helpful or accurate, and I've also used a more qualitative system for making defense strategy decisions.

Out of curiosity, why do you think the best defensive robots don't play defense during quals?

I would be happy too look at some of your matches. The reason why most good defensive bots don't play defense in quals is because they are usually some of the better offensive bots too. This years game tends allow good offensive teams to play effective defense very easily, all they need is a fast drive train and a good driver, which good offensive teams almost always have. I feel like our team demonstrated how easily the switch can be made from offense to defense at CVR when we spent all of eliminations playing counter-defense despite being one of the highest scoring teams there. We had a couple matches at other times (including the Sacramento regional) where we had to play defense and our driver was able to almost completely shut down the other teams offense with nothing but a drive train (they scored 0 tele-op points in several matches).

[dellagd]

The concept for a true DPR is to create a marker in this system that notates if a robot plays defense and upon whom they played it. We would then compare the amount a robot 'should score' aka their score in matches without defense to the score in their match where defense was played. A number would then be assigned to the defending robot for how many points or percentage of points a defender shut down.

Im implementing this, we'll see how it works.

Anything else?