Defense scouting

My team is looking into scouting the defensive ability of a team at a competition. In the past we have just recorded if a team has played defense. What do you all think is the best way to quantitatively evaluate the defensive ability of a team? What metrics should we record? Which ones are more important? Etc. Any thoughts are appreciated

2 Likes

generally defense isn’t something you can effectively track quantitatively, you could track forced misses or something like that, but the result of defense is effected as much by the bot getting defended as the defending bot. generally the most effective way of scouting defense that I have seen is just watching a team drive/play defense for a lot of matches and then pick them based on how good they are at defense

My favorite way to scout defense was explained to me by Karthik at his north champs strategy conference in 2019.

Record the team that plays defense, and WHO they play defense on. You can then take that team that is being defended and compare there score to their typical contribution when undefended.

So your defensive contribution would be their typical score vs when they are defended by the defense bot. You can then see how effective the bot is at defense

Simplifying:

Bot A plays defense on Bot B
Compare Bot B’s contribution typically to when they are defended by A

Repeat whenever A plays defense

Adding one more thing: record drivetrain and fouls. An 8-neo swerve bot is better than a 4 cim mecanum drive at defense 99% of the time, and fouls are the killer of defense.

14 Likes

If you want to be simplistic, rate defense on a 1-5 rating. 1 being poor defense and not having any impact on the match scores, 5 being amazing defense and having a significant/noticeable difference in match scores.

Defense is a hard thing to track but its important to note who is effective at playing the defense game.

This is also a great point, to see if teams can choose targets effectively. Adding this is a great idea if you want to be a little more in depth.

Obviously it depends on the game.

Some quantitative things you could record:

  • pinning (number of seconds that the defender pinned the opposing robot, number of times)
  • obstructing the path of an opposing robot (duration and number of occurrences)
  • the ability to push the opposing robot
  • number of shots that were missed due to interfereing with the robot’s shooting position

If the game has specific tasks that are valuable (such as hiding the ball in 2022) then you can keep track of that as well.

Probably the biggest metric with any defender is fouls. We have found that our choice for a defender often comes down to robots that are not going to commit fouls. So you definitely want to record that when scouting.

3 Likes

Here’s a thread from the past where this was discussed!

When it comes down to it, there isn’t really a great metric to zero in on. “Compare average undefended score to the score they got while getting defended” is a reasonable first approach, but it does still require some art. Small n is a pretty serious problem, even more so than it is already is in FRC. How do you evaluate the situation when either the defender or the defendee (or both) splits their time? If you’re dedicating the resources required to answer these questions satisfactorily, consider whether that solution beats the baseline of “have people who are good at watching FRC watch the matches and come up with an opinion.”

3 Likes

Defense is by far the hardest thing to scout for, as there is no measurable metric for how well a robot has performed defense. For us, we have a button that starts a timer, so we can get a general idea of how much time they spend playing defense, and then we have a notes section at the end of our system where scouts can write down notes from their robot.

This notes section is where we get the most value for measuring defense. This is the reason why we have our scouts as trained strategy members, instead of people who aren’t doing anything in the pit.

What we usually try to do is give an estimate of how well the defender was able to keep up with the robot that they are defending. We usually also try to give a metric of how fast/how much torque the robots drivetrain has. If we see a team that is pushing robots around, that is an important thing to note.

this is big, SEE IF THEY COMMIT FOULS. If a team keeps a team to 10 cycles under what they usually get, but fouls them so much that their defense results in no change in match score, there is no point in picking that robot. Also be wary of teams that have gotten yellow cards/don’t know about certain rules(e.g. protected zones) as you most likely don’t want to teach a team how to play by the rules while creating match strategies.

1 Like

this. Our team scouts even during weeks that we aren’t at a competiton, including week 0, to try to gain a better understanding of what a good robot/good defense robot looks like. Watching as many matches as possible is quite possibly the easiest way to increase the quality of your scouting data, especially if you have a notes/writing section in your scouting system.

1 Like

I agree with this, something that isn’t game specific though is tracking how many penalties that robot draws. Like if they get penalties called on them constantly they may not be the best defense bot if they are going to get a lot of fouls.

3 Likes

We don’t scout directly for defense. Instead we focus on driver skill and robot field performance. 1678’s Scouting White Paper and Fall Workshop presentations on scouting at www.citruscircuits.org describe our approach. We have added strategy notes on when teams play defense so we can go back and watch TBA videos on how effective it is, but the fact is that the best defenders often don’t play defense–they are driving so well that they are scoring instead. Focusing on driving ability and awareness is a better indicator.

17 Likes

This is definitely the way to do it. I think this is missed by many teams.

The best defenders I see do three things well.

  1. Play clean. They don’t get penalties, no matter how infuriating they are to their opponents.
  2. Play hard. They LOVE contact, not necessarily pushing matches but hits, turning their opponents, and pinning on occasion if they can do so.
  3. Take opportunities. They don’t always just stick to one robot, they defend anybody that happens to be handy then go back to their primary target.

And the best way to scout a defender is…
… Talk to your own drive team. Ask them which one team they do NOT want to have defending them. They may have 3 or 4.

9 Likes

This is a great input that hasn’t been in this discussion until now. Keep your friends close and your enemies closer.

7 Likes

Well, “defensive ability” is a pretty subjective thing to judge, because most of the data is qualitative. Things like “pretty good at blocking”, or “good driver” can’t be used in an equation if you were to calculate how “good” a team is. The best way I would try to assign a real number to this would be something like: “on a scale of 1-10, how effective was this team’s defense (if applicable)?”

That way you can realistically get an estimate on if the team was effective or not. Don’t try to make your scouters count how many points the defense denied from the opposing alliance, because there are far too many variables that can influence the number. If you had an alliance of two amazing offensive robots and one really bad driver, but the other alliance had one barely functioning shooter, would you call it good defense when the bad driver accidentally bumps into an opponent as they just so happen to be shooting? I would call that luck.

On the flip side, maybe don’t try to quantify strictly defense, and just try to get a 0 to 5 star rating of the driver as a whole? In theory a truly “good” driver should be able to be a formidable defensive bot, but what if they never get/have to play defense? If you were to take the most effective offensive player in the known universe, but never saw them play defense, would you completely discard that driver as a potential defensive option in case something were to go wrong? I’ve found that just rating the driver is a) easier on the scouters because it is a simple process that can be done post-match, and b) doesn’t rule out strong drivers just because they haven’t had to play defense.

TL;DR: I wouldn’t recommend strictly judging defense vs offense ability, but if you do, at least have an overall rating for them as well. And to somewhat quantify your data (albeit arbitrarily) you can do something like “on a scale of 0-10” or “0-5 stars” so you can at least get an idea.

In my experience thus far (3 blue banners in two seasons) with competitive driving, I can totally agree with this.

This past season, the robot I drove had an iffy-at-best intake with a nearly 100% accurate shooter, so when I could play offense I would. However, the intake was prone to failing a lot so I also spent a decent amount of time on defense (in fact, I was selected to the 2 or 3 seed alliance for defense skill; it also was a blue banner win). I’ll say that the deciding factors are mostly what you just laid out. No penalties, aggressive contact, and situational awareness. What I would add isn’t a new point, but a addition/revision to number 3. Designate a high-value target (the most powerful offensive robot), but only use that to influence decision making. Don’t only bully the HVT and take occasional shots at others just because. Have your drive coach feeding you what’s happening around the field that isn’t within your immediate vicinity. They should tell you things in concise callouts like “HVT aiming left” so you (meaning the driver) can quickly look left and see what’s happening. If your HVT hasn’t shot yet, you can go intercept them, but if they have you don’t waste time driving over to block a shot that already is in the air. The driver should only focus on the space immediately around the robot and it’s current target so they don’t miss that split-second evasion that the opponent tried, just because the defense driver was scanning the field. If you are actively defending a non-HVT, don’t break contact because your main target is up to something. Finish your shove or whatever on the other robot before moving to bully your primary target. The HVT designation should only influence what you do between moves, like if you had to decide between disrupting a lesser opponent’s cargo collection or your HVT’s cargo collection.

2 Likes

As well as if they get a foul or yellow card and its “out of character,” see if you can figure out why they got it. Some times that foul or yellow card it something they didn’t know was happening or the team themselves has no idea why they got it. (Aka check to see if you see someone from the team in the question box, if its a yellow card)
For example at Pease we got a yellow card for “damaging another teams intake” We were told we damaged the pneumatics on another teams bot. We didn’t even know that team had pneumatics. When we went to push them it was in our blindspot. We “knocked it loose” we had no idea how! We corrected after that and avoided the blindspot on the field.
So if you see one or two fouls, or a yellow card, see if they correct after that.

1 Like

This is a good point, I should alter what I say to if a team consistently draws fouls with no correction toward how they play. Whether or not they know the rule they’re breaking, if they can fix it then all is well but if they don’t then maybe not the greatest pick.

1 Like

This situation just wouldn’t come up for us. If there is a good offensive bot left available for picking, we’re taking them. Picking a defensive bot only comes into the picture when there are no good offensive bots left. In our mind a bot that’s scores N points on average is a better choice than one that might cut N points off of an opponent’s score. Perhaps someday there will be a game where this calculus changes. YMMV.

One strategy I’ve been fond of (I believe it was originally used by 2056): Have your three scouters of an alliance discuss after the match and rank the three robots on the alliance by driving ability. 3 points for the best driver, 2 for the middle, 1 for the worst. At the end of the competition you have point totals that help differentiate the best drivers.

2 Likes

We typically have a list of 8-12 offensive robots that we would pick before we get to a defensive bot, even if we turn that offensive bot into a defender. There have a few defensive standouts who broke into top offensive bots list.

1 Like