Your scouts hate scouting and your data is bad: Here's why

Competition Scouting
1

Hi, my name is Katie and I’ve been a Data Addict since 2010.

Alright so lets talk scouting - I love it, mostly because I love having the data afterwards, and I’m here because I have opinions about how to Get Gud at Scouting. Most teammates don’t love it and lets talk about this.

Why your scouts hate scouting and your data is bad:

  • You’re asking for too much data

    • Seriously consider what data you need - Not to assign an arbitrary number but if your scouting form has more than ~25 fields, you’re asking too much.

    • Condense - IE there are only 4 cargo that can be scored on a given level in the rocket - is there a way to combine or re-assign “how many” and “where” so you don’t have 9 different fields?

    • Make input easy and fast - the harder it is to record data, the worse your data is going to be. Scouting sheets should be large enough to be read easily and written in without tiny handwriting. Electronic sheets should have a small number of text-input and required fields.

  • You’re asking for the wrong data

    • This is an extension of the previous point

    • Look at the data you collected at your last event. Which columns did you actually use? What information did you wish you had? If their sandstorm is autonomous or not probably has little to no bearing on how your team will play with/against them but how well they drove might be useful if you’re looking for a defender.

    • Asking for data that you are not going to use devalues the work that goes into collecting said data.

  • You’re scouting for the wrong reasons

    • So, most teams aren’t picking. An event can have anywhere between ~30 - 60(?) teams and only 8 teams are picking. The other 22-52 still should scout. Why? Match strategy!

    • Consider scouting for your next match more important than scouting for pick lists -if you have to pick between only scouting teams you play with/against or everyone, the former is a better use of your team’s energy.

    • If you’re only scouting for pick lists and you don’t pick, your scouts feel like all that time was wasted. Using the data all the time makes the job relevant and valuable.

  • Your scouts don’t know why they’re doing this, aren’t engaged, and are “alienated from their product”

    • The drive teams know why they drive, same with pit crew, media, and chairman’s - they all see or work with the output of their efforts. Scouting often seeing their efforts go into a database and that’s it - they have no idea if its being used or if its busy work.

    • In general, team members want to add value to the team. If they don’t think scouting is adding value, why would they want to do it?

    • Scouts should be involved in preparing match strategy and pick lists - everyone should be invited to pick list meetings and the more I practice this the more firmly I stand on my beliefs (with rules, we’re not animals).

“But what about the sim-bucks strategy?” That is effective for 1114 because they came up with it and adapted it over the years and they clearly have a culture that encourages good scouting. Adding gimmicks and games to scouting may improve your turn-out but won’t improve your scouts engagement or quality of data. See the next point.

  • Your team’s culture around scouting sucks
    • Do you start scouting training with “I know scouting sucks but”?
    • Do you congratulate scouts after a match is won?
    • Do you threaten scouting as a punishment?
    • Do you thank your scouts for good data? Or only yell at them for bad data?
    • Every subteam has some perk/reward: what is scouting’s reward? (hint: it’s comfy chairs)(only applicable if you’re competing at a venue with bleachers)
    • Does anyone “important” scout? Does the drive team act as though scouting is “beneath them?” Are team captain’s scouting? Are team adults engaged in the process? (hint: everyone scouts! )

So - how to make scouting amazing and awesome? Here is my quick, actionable item shortlist:

  • Everyone scouts at least one match.
  • Scouts create strategy data dumps for matches. Scouts do this and not strategy/drive team because of the whole “seeing how your work is used” and not having that behind a curtain.
  • Give scouts special benefits - be it treats, comfy seating, or otherwise.
  • Scouts are directly involved in the pick-list process (if applicable)*.
  • Every competition sub-team gets a genuine shout out before, during, and after the event.

*“How to Pick List” is a completely different topic that I’ll have to touch on at anther time.

Thanks for coming to my TED Talk!

289 Likes
2

:clap::clap::clap:^

edit:
Hope we can draw from some of this and lessons we had this year. I would add, the scouting team needs a devoted mentor. I feel I failed their efforts this year, with other pressing issues taking my attention.

We bounced from a paper system that overstressed the scout lead the first regional (from data entry demands) to a pre-built phone scouting system (that while pretty awesome) was overdemanding on data collected and ended up overcomplicating the analysis. We could have done more if we’d spent a little more time testing it out. All in all, a lot of this applies, and I’ll be sure to share with the scouting group.

5 Likes
3

Preach

4 Likes
4

While I agree with most of this post, I disagree with the bit about ~25 fields.

If you have good reason for a lot of data AND the scouters understand why they’re collecting data AND the layout and ease of recording is good AND you’re in a picking position THEN they usually aren’t alienated. But if any of these aren’t true then it definitely sucks.

Having the data be useful in the end of the day is key. Not using it at all is certainly discouraging.

Even if you’re not in a picking position, having a “pre match” scouting sheet will help you form an alliance strategy. If you aren’t using your scouting data for this, you’re doing it wrong. Teams stretch the truth about their robot all the time and knowing their actual capability is very very important.

7 Likes
5

This is one of the best posts about scouting in years. Thank you.

11 Likes
6

This. 100% this. I was a scout during my first 2 years as a student on FRC and I definitely felt unappreciated at times while scouting. Not saying the team I was on had a poor scouting culture necessarily (it definitely did not), but it could have been better. Luckily, I loved scouting and data as well, but it definitely was a repetitive and draining task. Even though I was fortunate to be on a team that was large enough to have shifts of scouts, it was still grueling. I can only imagine how painful it is to scout without being interested.

I’ll add on this to a bit. It’s incredibly important that scouts are completely involved in this process because they are the ones watching all the matches. Data is a great way to summarize and solidify what they watched but at the end of the day they watched all the matches and they have a good feel for the competition.

It’s also important that you have an alliance selection representative that listens to and works with the scouts. Nothing is more demoralizing than having an alliance selection representative who does not make decisions representative of the data scouts collected throughout the competition and what the scouts felt themselves too. Trust me it feels like absolute crap knowing you spent all those hours in the stands scouting only to have the person making the decisions sway away from your work without consulting with the scouts or acknowledging their work. Speaking from experience my sophomore year of FRC. Wish I spoke up about it then, but I learned from it and hopefully became a better leader from it.

Looking forward to that post!

10 Likes
7

This is an absolutely fantastic post that I agree with wholeheartedly.

And if your team is interested in switching to an electronic scouting system, take a look at what our team has put together. Might not be for everyone, but is working great for us.

And we’re happy to share it with everyone!!

https://robostangs.com/scoutingapp

7 Likes
8

I think it is pretty natural to make the lead scout the person who is out there for alliance selection. Closes the circle. They know the pick list better than any other team member.

13 Likes
9

These points here are probably the most salient w/r/t scouting I have ever seen. Thank you for making this point.

I love to make scouts, particularly the student lead scout, feel like an all star. They absolutely should be on the field for alliance selection. The drive coach should defer to them for advice. They should be empowered to make decisions on which data points to collect and how to collect them. They direct the evening debriefs.

4 Likes
10

If those things are true, you probably aren’t the intended audience of the OP.

I agree with your comment but rules of thumb like that are just that; of course there is sometimes an exception.

4 Likes
11

This is super helpful, thanks!

12

I really like this. There’s also a lot of data that you may want–and you may even need–but you don’t need to collect. The FRC API (and the TBA API, conversely) provides a lot of this data, now more than ever. There should be a lot of data on this floating around but if anyone needs more info on how to find out climbing data, starting positions, no shows, and more without scouting, please ping me on here. No time now, but there’s lots of ways to cut your work and keep the superfluous amounts of data you want.

1 Like
13

This is a great one. We called out the work of our scouting lead at the end of our 2017 regional after we made it to the semi-finals in an upset of a higher seeded alliance. Our robot was nothing but a defender and the scouting data they had led us to two picks that carried our alliance to a 4 rotor match when we couldn’t score a gear.

Thanks for the reminder as we go into our event next week.

5 Likes
14

really interesting post, thanks for sharing, scouting is always one of those things so it’s always nice to see how to improve

1 Like
15

Scouting is essential to be competitve. It is also a lot of work and not necessarily something you can force on a HS student. Its not rocket science.

In any competition there are only several dozen teams, 24 of those in regionals compete in extended play.

The goals of scouting are three-fold: Use it to win quafication games, use it to build an alliance, lastly use it to hopefully beat other elimination alliances.

I get the simplicity of scouting…many don’t and there is a propensity I notice every year to come up with an app for that. I don’t believe there is an app for that. Never will be.

My reasons are similar to this threads OP…bad data (and small sample sizes)

There is not a single student that I have come across that sees the game like I do. They can learn observing and sort of get it and then turn around and over complicate it. My team right now is attemting to design an app/spreadsheet to “think like I do”…that is a waste of time IMO. Its about simplifying ,not over complicating it. Garbage in…garbage out.

Students do see the value in and do most of the observing ,Their efforts pay off we exteemly rarely missed elims (1 time) or in otherwords scouting led to extended playtime. It works. Its a force multiplier.

I am not against data is useful for a quick cross-event comparison and to validate your observations. As for in event improvement, data does not win games . Data is useful to validate. Nothing replaces observation and good notes. These notes need to be passionate and not just entries, written by someone who gets the goals of scouting.

Nearly every team in a competition should be describable in one sentence. From the description it should be evident if you want to partner with them and also how hard they are to go against. This description is cumulative then backed by capabilities and liabilities.

What you will find is only a subset fit into those you want to partner and those that may cause you trouble… at the end of the scouting goal is to know who can help you win it and who you will likely be up agianst and how to mitigate that threat . Notice I said “nearly”… there are those that dont get extended sentence descriptions. Not a partner choice nor a threat. rather noted in one or two words…like "slow "

Three phases: Pre-event, Day 1, Day 2 each has a distinct goal. You would need three apps to replace those phases and once again over complicate it with bad entered data. Think analog and not digital. Simplify it.

Look at music…CD/streaming are cheap and acceptabe, yet LP albums are purer.
The way most teams scout is similar…cd/easy which makes sense given we live in the digital age.

5 Likes
16

I’m a scouting mentor for 2199-Robolions. I’m sure we’re not that unique, but our approach is not to try to collect all the data. We use more of a “marketing” approach, looking for sampling instead of completeness, and we use a combination of objective and subjective measures. We use some simple Google Forms and Google sheets. Our objective questions are things like ‘how many hatch panels?’ ‘How many cargo?’ ‘How high?’ and some simple ‘subjective’ questions like ‘How fast?’ ‘How accurate’? Then we use a scoring mechanism based on what the kids say they’re looking for in a robot and try to get a good sampling, scouting each team more than once, with different people (for example ,this year, our robot is great at cargo and climbing and OK at hatch panels, so our drive team wants to pick robots strong at hatch panels). I teach the kids simple Google sheet formulas and they can build visualizations. The kids are usually really engaged. And, since they don’t have to scout everything, all the time, they stay engaged. Plus, the parents also get involved when they’re bored.
I’ve been doing this since SteamWorks. In every competition we’ve been in, whether we’re picked or not, based on our scoring, we can usually predict the stronger teams as a whole. Happy to share with anyone who’s interested.

6 Likes
17

Great post! There’s only one thing I disagree with.

Before our events while the drive team is practicing and the pit crew is making improvements, the scouting crew is training using the livestream from other competitions. If the drive team and pit crew would come up to the stands during a break between matches, they wouldn’t have the necessary training to scout properly.

At competition everyone has their job to do, and everyone’s job is important. I wouldn’t tell the pit crew to go drive the robot, or the awards team to go do the media team’s job. Similarly, I wouldn’t tell anyone from any other sub-team to come scout. That makes it seem like the scouters are “un-skilled workers” and anyone can do their job well, which is definitely not the case.

11 Likes
18

This is great. We finally improved our scouting for FRC this year by doing some of those very things. I think the most important part is to make the scouts realize that their work is valuable, and use their data every match. We ended up as an alliance captain for the first time last weekend and it was largely because of strategic decisions made using our scouting data during quals and discussions with the drive team and scouts. Having a system where they could see the data was also helpful (collected on paper, entered into spreadsheet w/ nice graphs). We also had a team-wide “pick list” meeting at the hotel on Friday night and then the scouts adjusted the list during Saturday morning matches. We were really happy with our picks as a 5 seed and just fell short advancing to the semis.

We didn’t have a scout on the field during alliance selection (it was a captain who had been working closely with the scouts) but that’s something we’ve done in FTC and would like to do in FRC.

Anyway, our scouts were really involved and told me that they felt like they were contributing valuable information to the team. That hadn’t been the case at some events in the past.

2 Likes
19

Every year we tell the story of 2015 Tesla, when we were picking for our alliance, we picked the WORST ranking team at the event for our 2nd pick, the crowd oooooh’d. We tell the team that it was because our scouting data said that they were the best team left that complimented our alliance and that it was because of scouting that we knew who to pick and why to pick them. We lost 2-1 in the finals as the 3rd alliance, but had the highest average score in the quarters and semis.

We even started doing a scouting test, similar to but less detailed than the driver’s test.
We test our scouting app week 0, and the weeks leading up to our first event.

The only thing I wish we did better is to stress to the team that the students should observe every match so when they see a bot that catches their eye (in a good way), they can go and ask that team to explain how they came up with the idea for that mechanism, so that next year the student will have a little more robotics knowledge.

5 Likes
20

I disagree with that statement, and I think our team’s record supports my statement. Having deep data has numerous times identify teams that we screened out with our observational, anecdotal biases. What we did on Curie in 2013 was in large part keyed by our scouting data. Perhaps our best most recent example was at Utah in 2018 when our scouting data pulled out 399’s big improvement over the last 6 matches and 1410’s hidden exchange play. Our first pick of 399 was a shock to everyone else because they had not seen the data–their observations had missed it.

There are few elite teams with gifted experienced mentors who can identify a sufficient depth of teams without relying on scouting data, but I can count those teams on one hand. I know of no student-led scouting endeavors that can do this simply because students can’t see enough events.

16 Likes