Researching scouting System

Hey I’m researching what team have the best scouting data or the best scouting System. So if you can drop a few team number I will really appreciate it.

1690

2 Likes

1678 has a few scouting white papers that are great.

8 Likes

A lot of teams come to mind with their good systems but here are some of my favorites within the last couple years: 118, 1706, 1690, 1706, 1918, 1983

All give great data but are very different from one another.

3 Likes

We completely overhauled our scouting system over the last 2 years. We’ll be releasing this year’s whitepaper very soon!

14 Likes

Speaking of, if anyone has recommendations for OPR calculators that would be great because I suck at math lol

2 Likes

Because it’s summer CD, I’m going to go ahead and make a bold statement that will probably backfire in some way. But here goes: If you’re using OPR, you’re not committed to scouting.

Here’s why: If you truly have the data, that is, you’re watching every match and logging robot actions, OPR is completely useless to you. By the time OPR is statistically relevant (it likely won’t at most competitions), you should already have better data to go off of. The only time OPR should be used is to fill in your intel gaps, but that’s not going to happen if you’re really scouting. If you have the data, you end up spending more time trying to figure out why a certain team’s OPR is so high, only to find out they had 2-3 matches with really, really good teams and we’re carried. I’ll get down off my soapbox now.

3 Likes

I don’t think so actually. It’s an information theory problem.

The amount of information contained in the FMS data from matches at a single event is too small to make predictive estimates of robot performance.

It doesn’t matter whether you use OPR or any other similar metric: if the FMS is your sole source of information, there’s no way to slice and dice it to get meaningful-enough conclusions to make match decisions.

OPR and friends are great to have available. And there’s a lot of fun to be had looking at the data. But for your match decision stuff, you need to inject more information into the system

Enter the scouting team.

Of course, still lots to be said about what the other info to be collected has to be, or how to merge it with FMS data, or how to draw meaningful conclusions out of it…

I could not disagree more. OPR is the first step for many to understand how data inferences can be applied to FRC.

Especially for teams that don’t have a large team and cannot have 6 scouts at all times, it’s imperative to use fms fields to determine team’s abilities.

Also, OPR doesn’t have to be a total metric, you can calculate the component OPRs which give you a pretty good idea of a robot’s abilities. Some games are better than others in this regard but overall, it’s not the worst; especially when you’re first starting out.

That’s not how OPR works. It’s how much better your teammates did compared to their average. So you’d have to be carried by top teams that did significantly better than they normally do which is fairly rare. This is another reason why component OPRs are so helpful.

Though I’d agree for the most part, if you don’t have enough people willing to properly scout, fms can give some great estimates. When I was drive coach during my senior year, I had the amazing crew of 0 scouts to help me. My team had about 10-12 people on it and with the pit crew we didn’t have enough people that were willing to scout for the entire competition.

What did I do? I used the fms reported values and calculated teams estimated contributions. After having a few teams finding out and talking to me, I found that my picklist and estimates were very similar to the top teams at the event.

Also, don’t overestimate your scouts. Most scouts struggle and report inaccurate data. So unless you have some way to account for this, you likely have much worse data then you think.

5 Likes

I’m interested in looking at OPR trends and averages over time and also how defense/offense affects it. Having concrete numbers is easier.

I gotcha! Especially if you lack the ability to inject additional info from the outside, no harm in using the FMS data to your advantage! Indeed, that’s where a lot of teams fall.

Where I’m purposefully hedging is the definition of “match decision”, and I suspect that’s where our perceived difference opinion falls.

The core I’m trying to get at is that someone developing a scouting system should expect an FMS-only solution to always be outperformed by a solution where a scouting team collects additional good data.

But, whether or not the additional data is needed for any particular strategy decision will really just depend on what that decision is.

1 Like

It is cool (and necessary) to have data but imo the conversation should be around 3 things:

  1. Collect what you actually NEED (i have definitely run afoul of this)
  2. Focus on data integrity
  3. Reduction of data so decisions can be made quickly

Following that up other goals are:

  1. Always be able to answer “why” when there is an outlier
  2. There are far better statistics out there than the mean. What does your strategy prioritize?
  3. Match by match (i.e. per match average scores) may be the wrong way to visualize/summerize.
  4. Near realtime data/strategy to the driveteam is insanely helpful. Especially when you don’t have access to a FRC galaxy brained Cory.

There are lots of good systems out there for collection, the key is almost always in the interpretation.

2 Likes

We used koronis for collection and then an excel sheet for analysis. Definitely the best scouting system if you are trying to throw it together the day of competition, but there are much better solutions if you have time.
1690’s solution this year was very impressive. 1678 has developed a reputation for very solid scouting. Many of the better scouting systems remain largely secret, but these two really stand out as the best ones that actually document their system for everyone. 1678’s papers in particular are great because they discuss how they analyze data.

1 Like

To be honest there are a few reasons that a scouter is giving you bad data. 1 they were poorly trained. 2 they don’t care at all. 3 yall are asking for the wrong data. also if you can’t trust your scouter they shouldn’t be scouting.

3 Likes

Before you plunge down this line, can you share what your team’s resources are for creating and using a scouting system? How many programmers do you have? Do you have a history of delving deeply into strategy? How many students do you have available to scout? Do you have funds to acquire data collection equipment and/or portable computers? Each of these answers determine your constraints and opportunities for developing your own scouting system.

1 Like

We got like 5 programmers. We got 16 to 18 available to scout. Also we do have the funds for data collection. And we do delving deeply into strategy before each new game.

Im also going to post are 2022 scouting system when i can.

You probably can take on the challenge of the most complex scouting systems with those resources. Feel free to reach out to our team for any advice or questions. We can set up a Zoom call too.

1 Like