Strategy Sub-Team

[Amit3339]

Last year we decided to open a new sub-team for strategy to analize the game in a better way. The goals were clear but it didnt go as plan.

Just wanted to ask how do other teams are running their strategy brainstroming, is it in a small group? the whole team? is there any format of discussion? and if a strategy group should have an influence on the mechanical desing(For Example: can burglarers or the feeder slides).

Thanks in advance :)

[MaGiC_PiKaChU]

In my opinion, there are 2 types of strategy...

1- Design strategy - this is your design team that try to find the best concept to maximize your team's performance. This year, they'd realize that the fastest can burglars win einstein, and that uncapped stacks are pretty much useless, so they will build a bot in consequence

2-Game strategy - This is your drive team, and people around it. They have to maximize the score by planning what they should do on the field. Who should fight who in the can battles, who should make hp/landfill, throw noodles or not, coop, play defense(not this year)

I don't think that a sub team doing only strategy is useful; it is already done inside other sub teams

[Amit3339]

In my opinion, there are 2 types of strategy...

1- Design strategy - this is your design team that try to find the best concept to maximize your team's performance. This year, they'd realize that the fastest can burglars win einstein, and that uncapped stacks are pretty much useless, so they will build a bot in consequence

2-Game strategy - This is your drive team, and people around it. They have to maximize the score by planning what they should do on the field. Who should fight who in the can battles, who should make hp/landfill, throw noodles or not, coop, play defense(not this year)

I don't think that a sub team doing only strategy is useful; it is already done inside other sub teams

The big problem with the mechanical team that I see is that they dont have enough time to sit and discuss on analyzing the game. Our can burglarer came up as a gimik but turned out to be very useful, there wasn't any special attention to how fast it should be. Were trying to avoid those mistakes by putting a special attention to the game elements and Priorities of mechanisems desings. Hope to hear some more replyes :)

[GeeTwo]

If you aren't already aware of Karthik's presentations on strategy, start there:

myself and Team 1114 will once again be delivering our seminar on Effective FIRST Strategies
.
.

Here are the threads discussing previous editions of this seminar:

www.chiefdelphi.com/forums/showthread.php?t=115843 (http://www.chiefdelphi.com/forums/showthread.php?t=115843)
http://www.chiefdelphi.com/forums/showthread.php?p=1156250#post1156250
http://www.chiefdelphi.com/forums/showthread.php?p=1054705#post1054705
http://www.chiefdelphi.com/forums/showthread.php?t=76778
http://www.chiefdelphi.com/forums/showthread.php?t=84958

Here's video from the 2013 version of this presentation:

https://www.youtube.com/watch?v=Apk_X-maRf8

Here's video of the TEDx talk that was inspired by many of the elements of this presentation:

http://www.youtube.com/watch?v=MfC3JdkEVgQ
.
.
If you're looking for a straight lecture on all the topics, I suggest you watch the webinars we did this fall as part of the Simbot Seminar Series (https://www.youtube.com/playlist?list=PLG_KOHBuXHNdxeFJBcio6sOEVMR821tkV).

As with any engineering project, the first step is to understand the problem. In this case, it means learning the game rules (including an analysis of scoring and the availability/opportunity/denial to score) and figuring out what actions are necessary for a robot to be successful. If you skip that step, most of the effort put into developing a strategy is going to be wasted.

[XaulZan11]

The big problem with the mechanical team that I see is that they dont have enough time to sit and discuss on analyzing the game.

Additionally, there are students and mentors on the buildteam who just hate sitting down and talking strategy and just want to start building. Forcing them to spend a few days analyzing the game leads them to become disinterested and frustration for those who value strategic discussion and plannng. I found it's best to just send them to start prototyping something right away.

[GeeTwo]

Additionally, there are students and mentors on the buildteam who just hate sitting down and talking strategy and just want to start building. Forcing them to spend a few days analyzing the game leads them to become disinterested and frustration for those who value strategic discussion and plannng. I found it's best to just send them to start prototyping something right away.

That makes sense, at least as far as prototyping subsystems. By the end of the Recycle Rush game reveal video, you could certainly have come up with some activities that you might want the robot to do:

make a stack of totes
put a recycle container on top of a stack of totes
put litter into an RC

For Aerial Assist, it would have been:

pick up the ball.
project the ball to 6+ feet in the air
catch the ball (truss catch)

It wouldn't have been a complete set for RR, and in each case, one of them was considered not worth the level of effort by most teams, but a few restless folks trying to figure out how to do these things would usually have been time well spent, presuming the strategy sessions were also taking place.

Another useful activity for these folks would be to begin building the practice field, or at least the key elements.

[EricH]

It wouldn't have been a complete set for RR, and in each case, one of them was considered not worth the level of effort by most teams, but a few restless folks trying to figure out how to do these things would usually have been time well spent, presuming the strategy sessions were also taking place.

Another useful activity for these folks would be to begin building the practice field, or at least the key elements.

I just want to expand on this a bit.

Sometimes, the strategy team will need something done to show how easy/hard something is. For example, is it easy or hard to pick a frisbee up off the floor? How about stack two totes? In those cases, a quick message from strategy to the restless mechanical team should be enough to keep mechanical interested (busy) for a few hours to a few days.

Example:
Strategy: "We think we want to do X. But we're not sure if it's a good use of time yet. Can we get a prototype?"
Mechanical: *a few hours of drilling, hammering, and sawing later* "We have a prototype for X, it's not working very well but has some promise."
Strategy: "Eh... we'll put that aside for now, what about Y?"
Repeat.

And any team that doesn't have key field elements (scoring platforms, chutes, and the like) is going to have trouble at some point. So getting those built should also be a top priority.

[MMmmDUDE]

How we do it on 2512:
We watch the kickoff video as a team. After the video we gather either at a team members home or a sponsors building. Our strategy department prints off multiple copies of the manual, studies it, and then gives a presentation on various aspects of the game such as every way to score, what fouls there are and the different aspects to the playing field. Team members can ask questions during the presentation so everyone has a good understanding of how the game works. After the presentation students and mentors are put into groups of 4 or 5 and they brainstorm robot designs. Then a representative from each group presents the ideas that the group came up with to the whole team. Designing the robot goes all through the day into the night and then more the next day until the build team has come up with a design they like.

The strategy department informs the build team about the important aspects of the game and recommends certain mechanisms but it is ultimately up to the build team to come up with the robot. The strategy department studies the game more and brainstorms possible scenarios. Also before competition they look for information online about robots that our attending the same regional as us.

At competition the strategy department is in charge of planning matches. They also take pictures of all of the robots in attendance, asks teams questions about their robot, and takes data on teams while they compete in matches. We use this information in a meeting that we hold the day before the playoff/elimination alliance selections. At the meeting we come up with a ranking list of who we would like to be aligned with and make sure the alliance selector knows what robot does what.

Hope this helps!

[MMmmDUDE]

How we do it on 2512:
We watch the kickoff video as a team. After the video we gather either at a team members home or a sponsors building. Our strategy department prints off multiple copies of the manual, studies it, and then gives a presentation on various aspects of the game such as every way to score, what fouls there are and the different aspects to the playing field. Team members can ask questions during the presentation so everyone has a good understanding of how the game works. After the presentation students and mentors are put into groups of 4 or 5 and they brainstorm robot designs. Then a representative from each group presents the ideas that the group came up with to the whole team. Designing the robot goes all through the day into the night and then more the next day until the build team has come up with a design they like.

The strategy department informs the build team about the important aspects of the game and recommends certain mechanisms but it is ultimately up to the build team to come up with the robot. The strategy department studies the game more and brainstorms possible scenarios. Also before competition they look for information online about robots that our attending the same regional as us.

At competition the strategy department is in charge of planning matches. They also take pictures of all of the robots in attendance, asks teams questions about their robot, and takes data on teams while they compete in matches. We use this information in a meeting that we hold the day before the playoff/elimination alliance selections. At the meeting we come up with a ranking list of who we would like to be aligned with and make sure the alliance selector knows what robot does what.

Hope this helps!

[pabeekm]

We’ve only been running the strategy team for the past two years, but it has been immensely helpful so far in letting us know where our decisions are coming from and what our biggest priorities are.

Our strategy team is a small group who practice modeling games in the preseason (I'm happy to say that it’s been getting more popular; as of this offseason we already have 4 underclassmen who have been meeting every week since champs to practice). While the strategy team is at work the first few days of build season, the rest of the team is either reading the rules and brainstorming their own strategies to pitch to the strategy team, handling basic administrative tasks (getting shipping orders in, making supply runs, etc.), or preparing to build field parts.

On our team we've been doing things a little differently; we strictly separate mechanism design and “strategy design”. The strategy team brainstorms, analyzes, and refines sets of game objectives the whole team will later debate. The group does not discuss mechanisms much beyond basic requirements because we want to focus on identifying the best sets of objectives before becoming invested in particular mechanisms that may not meet the best gameplay goals. After we narrow down to the top 3 or so options using mathematic models, the factors that make one strategy a better option than the other cannot be easily weighed (presence of certain partners, mechanism feasibility, etc.); such factors require the full experience of the entire team to effectively judge. After debating the top options as a full team, we hold a blind vote to determine our favored strategy. We then get to focus our prototype building around meeting the top strategy's specific gameplay requirements.

It can also helpful to not just set clear goals, but to also set your design/gampeplay objective priorities in various tiers. When you need to make a tradeoff, these priorities and their tiers should make your defining factors obvious. This is very helpful because when things do go wrong, it’s much easier to pinpoint what factors you were weighing and where your design considerations can be improved for the future (ex. for us this year we made several poor tradeoffs primarily because we kept overestimating weight; weight-based decisions will thus be more carefully made in the future).

[evanperryg]

Last year we decided to open a new sub-team for strategy to analize the game in a better way. The goals were clear but it didnt go as plan.

Just wanted to ask how do other teams are running their strategy brainstroming, is it in a small group? the whole team? is there any format of discussion? and if a strategy group should have an influence on the mechanical desing(For Example: can burglarers or the feeder slides).

Thanks in advance :)
Never have a full team strategy session. It just doesn't work. Whether it's design strategy, match strategy, or picklist-making, the only efficient way to work on strategy is to have a small group of highly experienced, dedicated students and mentors. Our strategy team adapts over the year. Design "strategy" is done mainly by leadership, very experienced students and veteran mentors. At events, our strategy/picklist team is simply the lead mentor, drive team, the two scouting leaders, and 1-3 scouters who have been very active at the event. Strategy students should have something else to do, as well. All of our strategy students have other roles on the team as well, including mechanical, electrical, or outreach/chairmans.

[IronicDeadBird]

Never have a full team strategy session. It just doesn't work. Whether it's design strategy, match strategy, or picklist-making, the only efficient way to work on strategy is to have a small group of highly experienced, dedicated students and mentors. Our strategy team adapts over the year. Design "strategy" is done mainly by leadership, very experienced students and veteran mentors. At events, our strategy/picklist team is simply the lead mentor, drive team, the two scouting leaders, and 1-3 scouters who have been very active at the event. Strategy students should have something else to do, as well. All of our strategy students have other roles on the team as well, including mechanical, electrical, or outreach/chairmans.

I'm reading this to say that only those on the strat team brainstorm ideas is this true?

[cadandcookies]

An exception might be made for a full team that is very small. I know whem I was on 2220 with ~110 people on the team between students and mentors, doing a full team strategy meeting was simply impossible. With my current team, 2667 (which has about 15 students and 7 mentors), it's much more likely that we'll try to get the entire team, or at least all the people interested in strategy, involved in our discussions at the beginning of build season.

[Citrus Dad]

Additionally, there are students and mentors on the buildteam who just hate sitting down and talking strategy and just want to start building. Forcing them to spend a few days analyzing the game leads them to become disinterested and frustration for those who value strategic discussion and plannng. I found it's best to just send them to start prototyping something right away.

We require everyone to participate in the build strategy at the beginning of the build season. After working on the overall concept for 1 or 2 days, we start working on individual components which satisfies the hands on folks for a while.

[Citrus Dad]

We have two presentations (slides + video) on our season strategy and scouting strategy. We are trying to get them up on our website.

We don't have "separate" strategy subteams. They are overlays from other subteams. We have the entire team in on the Kick Off design strategy for several days. This lasts about a day and an half. They we're off to prototyping.

Our scouting app team spends time developing methods to capture desirable traits for our first and second picks. In that process we find other strategy nuggets.

At competition, we have a student specifically tasked with setting match strategy based on analysis before the competition and updated with our scouting data on our smartphone app. Our scouting system kicks out our draft pick lists. We then have a smaller group of about half a dozen get together to set up our draft lists. That team is made up of mentors, lead scouts, match strategy and a pit scout.

[evanperryg]

I'm reading this to say that only those on the strat team brainstorm ideas is this true?

Robot ideas/strategic design ideas can come from anyone, but actual match strategy is done exclusively by the strategy team members. For the sake of simplicity, I will consider strategic design apart from event strategy. Our strategy "subteam" is students from any tehnical/business subteam that is interested in strategy, understands it well, and can communicate effectively in a scouting meeting. Scouters are any student not in the pit, who aren't presenting chairman's, who have been trained on our scouting app. Here's our hierarchy for the scouting team:

Lead Strategist
|-------------------------------------------------------------------|
Lead Scout |
|------------------------------------------| |
Match Scouters (~20ish) Qualitative Scouters(3-5) Pit scouters (1-2)


Lead strategist: Develops match strategy with the drive team. Collects data aggregated by the lead scout, and processes it with Tableau and MS Access. Manages pit scouters, collects information as they see necessary for developing strategy and picklists.Assembles the shift schedule for match scouting before each event.

Lead scout: Develops the match scouting interface, i.e. scouting app or paper sheets. Aggregates data from qualitative scouters and match scouters, to be collected by the lead strategist. Makes sure match scouters and qualitative scouters are focused, comfortable and are attending their shifts.

Match scouters: collect qualitative data using the scouting app. Work 2-3 2 hour shifts per event. Makes sure that the lead scout has collected their data before leaving their shift.

Qualitative scouters: dont have a schedule, but scout for 75-100% of matches out of their own enthusiasm. Collect qualitative data from matches, including notes about what a team did, why they did it, speculation on a team's strategic potential, etc. Extremely important in scouting meetings, as these are the students who see the most matches, and can provide excellent feedback on almost any team.

Pit scouters: collect general technical information on every team at the beginning of the event. They also do not have a predefined schedule, but are called upon by the lead strategist when they are needed. They will be frequently called upon Saturday morning to get last-minute, very specific info needed to make the picklist as strong as possible. Our picklist meetings include the lead mentor, drive team, lead strategist, lead scout, and qualitative scouters.

[connor.worley]

1538 ran a similar model that worked well.

18-20 scouts on rotation collecting match data.

1-2 scout masters on rotation compiling the data in excel / getting relevant numbers for the strategy lead.

1 strategy lead who creates match strategies and relays them to the drive coach.

The drive coach makes sure the strategies are executed properly, but has minimal strategic input. Really takes a lot of pressure off the drive team. We'd go to meet with other drive teams before matches, but our strategy lead would come down as well and usually be the one leading the discussion.

[Lil' Lavery]

18-20 scouts on rotation collecting match data.

1-2 scout masters on rotation compiling the data in excel / getting relevant numbers for the strategy lead.

Somewhere out there, my scouting lead [now co-captain] is salivating at the thought of having that much manpower.

[Citrus Dad]

Somewhere out there, my scouting lead [now co-captain] is salivating at the thought of having that much manpower.

We did it with 8 scouts, 1 onsite scouting app programmer, 1 match strategy manager, and 1 pit scout. We had 3 mentors supporting this, plus the drive coach. We added scouts as we could to relieve the burden, but our minimum crew was highly functional.

[evanperryg]

Somewhere out there, my scouting lead [now co-captain] is salivating at the thought of having that much manpower.

It's a nice benefit of having a large team. Even the best match scouters get tired and sidetracked after a while, so switching them out can make people happier, and data more reliable.

[gblake]

Ask yourself (or the team) who is going to consume the ___ group's outputs (fill in the blank with "strategy subteam" or with any other group's name).

If no one wants, or if no one will use, what that ____ group produces, someone's time is being wasted (or several someones' time). That leads to all sorts of predictable results.

So, if you look through this end of the telescope, no matter how your team has been organized or will be organized, you don't need a strategist until someone says "I need the answer to X, and I think the answer depends on the strategy we are going to use." At that point, and not before, you need an individual or group able to answer that strategic question competently.

Having a gut feeling that "strategy" is a good thing is very different from actually using the results of developing a strategy. Also, what someone needs (because someone else says they need it) is not the same thing as something they will actually use.

In some teams, strategic geniuses and the results they produce would be totally wasted, because no one feels the need to have (someone else's) strategic thinking affect what they are doing. In other teams, a good strategist is considered worth their weight in gold.

I'll bet that if you assign team members to jobs using this sort of thought process, and iterate a few times, you will eventually be happy with the result. You will have people producing the answers or the items that your team is actually going to use to accomplish the things your team wants to do.

Blake
PS: I am a definite believer in translating a selected strategy into requirements, and then turning those requirements into a design. I also know it is a hard thing to do to get a herd of cats to understand the differences between those three things, and actually follow a process that keeps them separate and in-order. However, I think the payoff is worth the investment.

[Kevin Leonard]

Strategy is interesting, because I've seen so many teams run a "Strategy team" successfully in dozens of different ways.
The commonalities are:

A well-trained scouting team that takes quantitative and qualitative data and puts the data in a format that is easy to interpret.

A smart, strategic drive coach who knows how to make smart, split-second decisions on the field.

An understanding of different factors contributing to alliance make-up and picklisting (consistency, standard deviation, scarcity, differences between quals, quarters, semis, finals, etc.)

An understanding of the different ways to maximize an alliance's potential, or to consistently score at some threshhold.

A team that understands what it takes to win and is willing to take whatever steps it takes (making robot modifications, playing the back-up robot and having your own team sit out, etc.)

And perhaps most importantly, Gracious Professionalism at all times! No alliance can succeed without this, even if they have the best strategy and robots to back it up.

[Michael Hill]

Somewhere out there, my scouting lead [now co-captain] is salivating at the thought of having that much manpower.

Just do what we do...develop a scouting platform and give other teams access to the data. Our team doesn't have nearly enough scouters to do it ourselves, but because we're open with the data and analytics (and the software is decently good quality), we are able to get help from other teams at the regional (coordinated ahead of time).

[Lil' Lavery]

Don't interpret my comment to mean 1712 hasn't managed to scout with more limited manpower. Our finalist run at Chestnut Hill this year was largely powered by our alliance selections. We won the GP Award at the Upper Darby district in part because of how we helped other teams with their pick lists. We often pair up with another team to help gather data.

It was just a statement of jealousy towards the teams with sufficient manpower to do more advanced things with their scouting while not burning out their scouts.

[rick.oliver]

First, develop your game plans, call it your strategy, which describes exactly what you want to accomplish in each phase of the game based on your analysis of the game rules (scoring options, constraints, etc.). Prioritize the strategies and elements within each strategy.

That has always been a full team exercise on teams I've been associated with. The team size has always been less than 50 and greater than 20.

Second, develop design concepts which could deliver your strategy, evaluate (this is the engineering) and define your design basis. Again, mostly a full team exercise.

At this point begin conceptual design, prototyping in groups.

You may need to redefine your strategy throughout the build process as you learn or need to compromise ... refer to your priorities.

Once your design is firm and you've locked on your machine's capabilities, begin to define your scouting needs, strengths and needs for help from an alliance partner.

[Citrus Dad]

PS: I am a definite believer in translating a selected strategy into requirements, and then turning those requirements into a design. I also know it is a hard thing to do to get a herd of cats to understand the differences between those three things, and actually follow a process that keeps them separate and in-order. However, I think the payoff is worth the investment.

This explicitly the process we use the first few days of build season. We shout down design ideas the first day!

[Citrus Dad]

Just do what we do...develop a scouting platform and give other teams access to the data. Our team doesn't have nearly enough scouters to do it ourselves, but because we're open with the data and analytics (and the software is decently good quality), we are able to get help from other teams at the regional (coordinated ahead of time).

Our alternative is to develop a platform that only involves input from the scouts and virtually no data processing on site. Our tablet system is all programmed to do the data analysis on a remote server and then spit out the results. That makes our scouting much less stressful. (Except when the system glitches, but then that's the app programmers' headache.) We have 2 whitepapers up on setting up the system and will prepare another soon.

[Michael Hill]

Our alternative is to develop a platform that only involves input from the scouts and virtually no data processing on site. Our tablet system is all programmed to do the data analysis on a remote server and then spit out the results. That makes our scouting much less stressful. (Except when the system glitches, but then that's the app programmers' headache.) We have 2 whitepapers up on setting up the system and will prepare another soon.

We do the same...we just take in A LOT of data. We've recorded positions of events since 2012. That's knowing where teams are scoring (with heatmaps), fouls, and defensive efforts. We have 4G tablets and have teams come help us.

Here is the 2013 page: http://innovators3138.org/dash/

Select either Queen City or Crossroads (We did IRI, but I don't see the data there for some reason). Select a match. The teams should load. There are checkboxes next to each of them. Select a team you're interested in and whether you want made shots, missed shots, and whether in tele or autonomous.

[Amit3339]

We have two presentations (slides + video) on our season strategy and scouting strategy. We are trying to get them up on our website.

We don't have "separate" strategy subteams. They are overlays from other subteams. We have the entire team in on the Kick Off design strategy for several days. This lasts about a day and an half. They we're off to prototyping.

Our scouting app team spends time developing methods to capture desirable traits for our first and second picks. In that process we find other strategy nuggets.

At competition, we have a student specifically tasked with setting match strategy based on analysis before the competition and updated with our scouting data on our smartphone app. Our scouting system kicks out our draft pick lists. We then have a smaller group of about half a dozen get together to set up our draft lists. That team is made up of mentors, lead scouts, match strategy and a pit scout.

Can you give more information about scouting strategy? I'm not sure I fully understand it's meaning

[Citrus Dad]

We do the same...we just take in A LOT of data. We've recorded positions of events since 2012. That's knowing where teams are scoring (with heatmaps), fouls, and defensive efforts. We have 4G tablets and have teams come help us.


We tried heat maps in the first 2014 regional and it was to overwhelming for the scouts and we lost accuracy elsewhere. So we cut back to just counting type data.

[Citrus Dad]

Can you give more information about scouting strategy? I'm not sure I fully understand it's meaning

The scouting strategy is about in-competition strategy. We use it for both match strategy and draft picks. We need to put up our Powerpoint to explain it more fully. I'll see if we can get it up on the white papers here.

[Michael Hill]

We tried heat maps in the first 2014 regional and it was to overwhelming for the scouts and we lost accuracy elsewhere. So we cut back to just counting type data.

Yeah, it is pretty difficult to do well. However, it provides so much valuable information. For example, in 2013, forcing a team to shoot from a spot where they make the fewest shots. You can also find the routes that teams travel through as well and are able to disrupt them enough to lose a cycle or two.

[evanperryg]

Yeah, it is pretty difficult to do well. However, it provides so much valuable information. For example, in 2013, forcing a team to shoot from a spot where they make the fewest shots. You can also find the routes that teams travel through as well and are able to disrupt them enough to lose a cycle or two.

That can be much more easily done with qualitative data. Good qualitative scouters can produce that information and a whole lot more, without polluting quantitative data.

[Michael Hill]

That can be much more easily done with qualitative data. Good qualitative scouters can produce that information and a whole lot more, without polluting quantitative data.

Using qualitative data is fine...if you trust the person/people taking down the information consistently (which rarely ever happens). Having actual data is very useful for the drive team to have at hand when discussing strategy with other alliances. It really takes the "Yeah, we can do [x] every time. Let us do that." When we can say "Hmm...let me show you what you really do. Now lets win the match this way...".

[evanperryg]

Using qualitative data is fine...if you trust the person/people taking down the information consistently (which rarely ever happens). Having actual data is very useful for the drive team to have at hand when discussing strategy with other alliances. It really takes the "Yeah, we can do [x] every time. Let us do that." When we can say "Hmm...let me show you what you really do. Now lets win the match this way...".

I'm not saying rely entirely on qualitative data. I'm saying qualitative data streamlines the more detailed data into something much more usable. I only assign the most experienced, most knowledgeable scouters to qualitative scouting. If you can't trust your own scouters to be taking decent data, either they need more training or they just shouldn't be scouting.

[jajabinx124]

I only assign the most experienced, most knowledgeable scouters to qualitative scouting. If you can't trust your own scouters to be taking decent data, either they need more training or they just shouldn't be scouting.

I agree with how you assign your scouts. It takes more skill, more game knowledge, etc. to scout qualitatively than quantitatively. Quantitative data is just data scouts are gathering that requires basic game understanding(Where scouts are collecting data based of dos and don'ts of a robot and numerical data). Qualitative data asks the scout to summarize the robot's performance and role during a match that the certain robot played during a match(Which IMO takes a deeper understanding of the game and understanding of important roles in a game), so it's a smart idea that a team should assign their more experienced scouts to this role.

You bring up another good point about trusting your scouts to take good data or not. How do you guys train your scouts if it comes to that point?

[evanperryg]

You bring up another good point about trusting your scouts to take good data or not. How do you guys train your scouts if it comes to that point?

Generally, we have finished the entire scouting system by week 4 of build, and only make minor changes after that point. So, during week one of competitions, prospective scouters come to a Saturday morning meeting where they learn to scout by watching the live stream of some week one event. This is a great time to assess how well the system works, and to see who on the team should be a scouter at events. Generally, qualitative scouters are students that did quantitative for an entire prior season, who did an outstanding job and have a very strong understanding of the game. Qualitative scouters require no special training, just occasional guidance from one of the scouting leads as to what they should look for. What we ask them to look for can vary event to event or even hour to hour, but there are a few basic things that they always know to look for- deadbots, repeated and egregious penalties, things like that that almost immediately put any team on the DNP come Friday evening.

[gblake]

To those of you familiar with how your own teams do scouting, do the scouts carry out specific evaluations defined/requested by strategists?

Given that scouts could measure/record an infinite number of things, is a strategist explicitly the person who narrows those possibilities down to what is actually collected?

[evanperryg]

To those of you familiar with how your own teams do scouting, do the scouts carry out specific evaluations defined/requested by strategists?

Given that scouts could measure/record an infinite number of things, is a strategist explicitly the person who narrows those possibilities down to what is actually collected?

We have one "strategist," assuming your definition of strategist is "that guy that tells the drive team what the alliance probably should do." But yes, especially Friday evening or saturday morning I will ask a pit scouter to go ask a couple teams some very specific questions. Examples of this season's very specific questions include "how quickly can you upright a container?" and "do you /really/ need your ramp in order to load from the feeder?"

[Gregor]

We have one "strategist," assuming your definition of strategist is "that guy that tells the drive team what the alliance probably should do." But yes, especially Friday evening or saturday morning I will ask a pit scouter to go ask a couple teams some very specific questions. Examples of this season's very specific questions include "how quickly can you upright a container?" and "do you /really/ need your ramp in order to load from the feeder?"

We have the exact same system. Lead scouting mentor and student come down and discuss with our drive team what our alliance should be doing. Time permitting, they come to the pre-match alliance meeting, but are normally quiet unless they drive coach misses something (which isn't uncommon, I miss things :rolleyes: ).

[Citrus Dad]

Using qualitative data is fine...if you trust the person/people taking down the information consistently (which rarely ever happens). Having actual data is very useful for the drive team to have at hand when discussing strategy with other alliances. It really takes the "Yeah, we can do [x] every time. Let us do that." When we can say "Hmm...let me show you what you really do. Now lets win the match this way...".

When we first put up the smartphone app for our drive team, we called it the "b---s--- detector.":yikes:

[Citrus Dad]

To those of you familiar with how your own teams do scouting, do the scouts carry out specific evaluations defined/requested by strategists?

Given that scouts could measure/record an infinite number of things, is a strategist explicitly the person who narrows those possibilities down to what is actually collected?

I think a better term for what we call a "strategist" really is a match tactician. They're like a basketball assistant coach or football offensive coordinator who puts together the in-match strategy. They use data from the scouting app to identify the strongest teams on both sides and if a particular team can carry out a task. They give some feedback on the stats after the competition so that we can update the scouting app.

[Michael Hill]

When we first put up the smartphone app for our drive team, we called it the "b---s--- detector.":yikes:

I wanted to have a "BS Factor" put in that reflected the difference of what a team said they could do in pit scouting versus what they actually do. The programmers were about to do it, but since we were sharing with other teams, we decided we probably shouldn't.

[Citrus Dad]

Generally, we have finished the entire scouting system by week 4 of build, and only make minor changes after that point. So, during week one of competitions, prospective scouters come to a Saturday morning meeting where they learn to scout by watching the live stream of some week one event. This is a great time to assess how well the system works, and to see who on the team should be a scouter at events. Generally, qualitative scouters are students that did quantitative for an entire prior season, who did an outstanding job and have a very strong understanding of the game. Qualitative scouters require no special training, just occasional guidance from one of the scouting leads as to what they should look for. What we ask them to look for can vary event to event or even hour to hour, but there are a few basic things that they always know to look for- deadbots, repeated and egregious penalties, things like that that almost immediately put any team on the DNP come Friday evening.

Our system is similar. We wish we had it built by Week 4...:o

[artK]

I wanted to have a "BS Factor" put in that reflected the difference of what a team said they could do in pit scouting versus what they actually do. The programmers were about to do it, but since we were sharing with other teams, we decided we probably shouldn't.

Why not just not get data points that you can't verify in the pit? It saves time and reduces the amount of noise in your data.

[EricH]

Why not just not get data points that you can't verify in the pit? It saves time and reduces the amount of noise in your data.

That's kind of the point of the "detector". If you aren't at least looking in the pits, it's kind of odd... but it's not that hard to find a team that is using favorable numbers when you do.

For example, let's say your pit scouts talk to a team about their stacking. And they say that they can do two 6-stacks, uncapped. Sounds reasonable, right? I can see teams going for that this year. But what YOU don't know--necessarily--is that that's their design/non-interference/non-spec (and, in non--RR games, non-defense) field number. The real number, as your scouts find by observation, is that they're actually doing 1 6 stack, or maybe 3 3-stacks, or one 4-stack and one 2-stack.

Basically, it's not the data points that you can't verify that you get from the pit that are the problem--most teams can verify anything they get from the pit scouts. It's that many teams will be optimistic--I don't accuse them of lying outright--but this is a physics problem. (Read: Equations (gameplay) work out nicely in the theoretical frictionless vacuum world, not quite so well in reality.) Then the verification shows that optimism to be misplaced.


My opinion on pit scouting is actually a bit different than most people's. My personal opinion is that it's NOT about the robot when you're scouting in the pit. The only thing you do with the robot is to take a picture (remind the field scouts what this robot looks like), and maybe note what work is being done. The TEAM is far more important, in terms of getting to know them. Let the match scouts collect the data on whether the team works on the field: if two teams work well together and are good friends, and both have decent robots, look out in eliminations if they're on the same alliance.

[IronicDeadBird]

Robot ideas/strategic design ideas can come from anyone, but actual match strategy is done exclusively by the strategy team members. For the sake of simplicity, I will consider strategic design apart from event strategy. Our strategy "subteam" is students from any tehnical/business subteam that is interested in strategy, understands it well, and can communicate effectively in a scouting meeting. Scouters are any student not in the pit, who aren't presenting chairman's, who have been trained on our scouting app. Here's our hierarchy for the scouting team:

Lead Strategist
|-------------------------------------------------------------------|
Lead Scout |
|------------------------------------------| |
Match Scouters (~20ish) Qualitative Scouters(3-5) Pit scouters (1-2)


Lead strategist: Develops match strategy with the drive team. Collects data aggregated by the lead scout, and processes it with Tableau and MS Access. Manages pit scouters, collects information as they see necessary for developing strategy and picklists.Assembles the shift schedule for match scouting before each event.

Lead scout: Develops the match scouting interface, i.e. scouting app or paper sheets. Aggregates data from qualitative scouters and match scouters, to be collected by the lead strategist. Makes sure match scouters and qualitative scouters are focused, comfortable and are attending their shifts.

Match scouters: collect qualitative data using the scouting app. Work 2-3 2 hour shifts per event. Makes sure that the lead scout has collected their data before leaving their shift.

Qualitative scouters: dont have a schedule, but scout for 75-100% of matches out of their own enthusiasm. Collect qualitative data from matches, including notes about what a team did, why they did it, speculation on a team's strategic potential, etc. Extremely important in scouting meetings, as these are the students who see the most matches, and can provide excellent feedback on almost any team.

Pit scouters: collect general technical information on every team at the beginning of the event. They also do not have a predefined schedule, but are called upon by the lead strategist when they are needed. They will be frequently called upon Saturday morning to get last-minute, very specific info needed to make the picklist as strong as possible. Our picklist meetings include the lead mentor, drive team, lead strategist, lead scout, and qualitative scouters.

Ah that makes sense the strategy activity is during comp I take it.

[artK]

That's kind of the point of the "detector". If you aren't at least looking in the pits, it's kind of odd... but it's not that hard to find a team that is using favorable numbers when you do.

My opinion on pit scouting is actually a bit different than most people's. My personal opinion is that it's NOT about the robot when you're scouting in the pit. The only thing you do with the robot is to take a picture (remind the field scouts what this robot looks like), and maybe note what work is being done. The TEAM is far more important, in terms of getting to know them. Let the match scouts collect the data on whether the team works on the field: if two teams work well together and are good friends, and both have decent robots, look out in eliminations if they're on the same alliance.

I'm not suggesting looking at the pits at all. My opinion about pit scouting is: get a good picture, get a look at the drivebase (just to get an idea of what there driving might look like), record important dimensions (can the robot fit under a pyramid, does it drive long or wide when balancing). It got all the information we could want that match scouts usually can't get by watching a match, without any risk of memory biases being introduced into your data, and was relatively simple (a championship divisions worth of pit scouting could be kept on like 4 sides of paper).

[jajabinx124]

It got all the information we could want that match scouts usually can't get by watching a match

The definition of pit scouting in my book is asking teams questions about their robot that can't be answered through match scouting. Limit pit scouting to data your actually going to use(anything that can't be answered through match scouting) or in other words like you said, keep it simple(actual questions can be complex, but not redundant to what your match scouters are observing). Don't ask teams questions you can answer through match scouting because It'll be a waste of time and It'll be a pain on Friday night going through redundant pit scouting data.

[Citrus Dad]

The definition of pit scouting in my book is asking teams questions about their robot that can't be answered through match scouting. Limit pit scouting to data your actually going to use(anything that can't be answered through match scouting) or in other words like you said, keep it simple(actual questions can be complex, but not redundant to what your match scouters are observing). Don't ask teams questions you can answer through match scouting because It'll be a waste of time and It'll be a pain on Friday night going through redundant pit scouting data.

I agree. The last two years we have done extensive pit scouting on a couple of elements that couldn't be discerned from the stands. Last year it was pass through time and reliability because many teams weren't doing this enough to get a good measure (and we walked around with a watch and clipboard). And this year of course we were looking for the ability to add cangrabbers. Perhaps the biggest plus was being able to get to know teams about a specific issue. We both got to know how amenable they were and also just simply connected. We now intentionally try to come up with a specific pit scouting task like this.

[EricH]

I'm not suggesting looking at the pits at all.
"Why not just not get data points that you can't verify in the pit?" Guess you're a stereotypical English-challenged engineer; that's a multi-negative and an unclear prepositional reference in the same sentence. What I initially read was something more like: "Why not just get data points in the pit that you can verify?" If I could rewrite the initial statement to what you meant: "Why not only get data when you're pit scouting that you can't verify on the field?"

And if that is what you meant, then my response is: What data do you mean?

Just about any key data point that you could pick up in the pit--like the ones you mentioned--can also be picked up on the field, given time. The biggest difference is that if you get it in the pit area, you might get it faster (given that teams don't always play all their cards on the field at any one time).

That being said, I'm sure there are exceptions to that: inter-team dynamics, and what improvements they're planning on (or can be persuaded to adopt), are two of the ones that come to mind. Oh, and the all-important one for a top seed: What the odds of them accepting your selection are...

[Michael Hill]

I agree. The last two years we have done extensive pit scouting on a couple of elements that couldn't be discerned from the stands. Last year it was pass through time and reliability because many teams weren't doing this enough to get a good measure (and we walked around with a watch and clipboard). And this year of course we were looking for the ability to add cangrabbers. Perhaps the biggest plus was being able to get to know teams about a specific issue. We both got to know how amenable they were and also just simply connected. We now intentionally try to come up with a specific pit scouting task like this.

Ya, we had the weirdest pit scouting questions last year (out of pit scouting questions we've been asked). They were mostly about drive team experience. For example, we asked how much experience the drivers had, how much practice time they had with that particular robot and drivetrain. Took note whether or not the coach was a student or mentor and how many much experience they had. A lot of that is valuable information. For example, our drive team is 100% students. However, when discussing strategies, there were numerous occasions where an adult coach would "bully" the other two student coaches into playing their strategy. If we noted that their coach is a mentor, we also send a mentor to the strategy sessions; not to really participate, but to maintain a presence. We've found that if another mentor is present, those problems tend to go away. We also ask questions about what the intent of the robot (specifically worded to extract intent of the design). That way, we are able to juxtapose that with how they are doing (and if they are performing to expectations).

[artK]

Guess you're a stereotypical English-challenged engineer;


Yeah. I know I am not a good writer (I usually revise my posts a fair bit to compensate for this, like five minutes just writing this sentence out).


And if that is what you meant, then my response is: What data do you mean?

Just about any key data point that you could pick up in the pit--like the ones you mentioned--can also be picked up on the field, given time. The biggest difference is that if you get it in the pit area, you might get it faster (given that teams don't always play all their cards on the field at any one time).

That being said, I'm sure there are exceptions to that: inter-team dynamics, and what improvements they're planning on (or can be persuaded to adopt), are two of the ones that come to mind.

The rationale I have for pit scouting being really simple is because pit scouting usually happens during practice day, and a lot can change between then and drafting. But some things, like a drivebase or height, usually don't change between those points, or are changed for a match for some specific strategy.

With respect for inter-team dynamics and improvements, most of that analysis can be done informally by the drive team and pit crew in preparation for matches, especially since they'll be the ones working directly with them during matches.

[GeeTwo]

The rationale I have for pit scouting being really simple is because pit scouting usually happens during practice day, and a lot can change between then and drafting. But some things, like a drivebase or height, usually don't change between those points, or are changed for a match for some specific strategy.
Pit scouting is also essential for strategy in the first few rounds, especially if a robot didn't play many practice rounds, or calibrated and tested during practice rounds rather than competing. If you base your strategy on an assumption that your adversaries can do 90% of what pit scouts report, and your allies can do 50% of what pit scouts report, you'll usually do better than if you start from scratch.
It may also help the field scouts select which robot each will watch. For example, if Bill and Carol watch most of the landfill 'bots and Mary, Frank, and Ted watch most of the feeder station 'bots, results are likely to be more consistent than if assignments are random.

[gblake]

I'll bet EricH and ArtK would quickly meet in the middle if they both agreed scouts should collect data that drivers, etc. want and will actually use; and that scouts shouldn't waste time collecting any other data (unless they have time to waste); and that scouts should get the needed data from wherever it is available, regardless of where that is.

If you need data from the pits, go get it. If you don't, don't do scouting there.

People shouldn't have the job of pit scouting, they should have the job of getting needed data; if you know what I mean.

[Kevin Leonard]

I'll bet EricH and ArtK would quickly meet in the middle if they both agreed scouts should collect data that drivers, etc. want and will actually use; and that scouts shouldn't waste time collecting any other data (unless they have time to waste); and that scouts should get the needed data from wherever it is available, regardless of where that is.

If you need data from the pits, go get it. If you don't, don't do scouting there.

People shouldn't have the job of pit scouting, they should have the job of getting needed data; if you know what I mean.

I agree with you for the most part, but sometimes its difficult to tell at the beginning of the season exactly what data is going to be the most important.

For example, in 2014 Team 20 thought that the amount of time the robot possessed the ball was going to be extremely important, and that our second pick would be largely based on that. As it turned out, that was completely unimportant, and our match scouts did far more work than necessary getting that information, and things like "What drivebase does Team X have?" and "How smart are their drivers and how well do we work with them?", both of which rely at least partially on pit scouting.

However, assuming you have the manpower to make it happen (not something Team 20 lacks), it's always better to have more data than not enough.

If you don't have the manpower to make it happen, either team up with another team so you gain the manpower, or find a way to narrow the data collection to only what is essential.

[GeeTwo]

I'll bet EricH and ArtK would quickly meet in the middle if they both agreed scouts should collect data that drivers, etc. want and will actually use; and that scouts shouldn't waste time collecting any other data (unless they have time to waste); and that scouts should get the needed data from wherever it is available, regardless of where that is.

Yes, I concur that this thread has reached the point of (verbally) violent agreement.

[IronicDeadBird]

*INCOMING CONTROVERSIAL STATEMENT*

I'm truly surprised at how intensive some of you guys are with scouting during competition. I mean to be fair this year was the first time I was faced with having to make a pick list. But scouting didn't effect match to match strategy as much as Q'ing up did...

[connor.worley]

*INCOMING CONTROVERSIAL STATEMENT*

I'm truly surprised at how intensive some of you guys are with scouting during competition. I mean to be fair this year was the first time I was faced with having to make a pick list. But scouting didn't effect match to match strategy as much as Q'ing up did...

2nd round picks win worlds...

[IronicDeadBird]

2nd round picks win worlds...

Well that and caffeine...

[Kevin Leonard]

*INCOMING CONTROVERSIAL STATEMENT*

I'm truly surprised at how intensive some of you guys are with scouting during competition. I mean to be fair this year was the first time I was faced with having to make a pick list. But scouting didn't effect match to match strategy as much as Q'ing up did...

Smart strategy and scouting can take a weak robot and get it into eliminations.
Smart strategy and scouting can take a mediocre robot and make it a regional contender.
Smart strategy and scouting can take a decent robot and turn it into a regional lock.
Smart strategy and scouting can take a good robot and make it a sure bet to win a regional.
Smart strategy and scouting can take a great robot and bring it to einstein.
And smart strategy and scouting is essential for an elite robot to win a World Championship.

This year, 20 had the second one. Last year, 20 had the fourth one.
The goal is to one day strike lightning on robot design so we can be the sixth one.

[IronicDeadBird]

Smart strategy and scouting can take a weak robot and get it into eliminations.
Smart strategy and scouting can take a mediocre robot and make it a regional contender.
Smart strategy and scouting can take a decent robot and turn it into a regional lock.
Smart strategy and scouting can take a good robot and make it a sure bet to win a regional.
Smart strategy and scouting can take a great robot and bring it to einstein.
And smart strategy and scouting is essential for an elite robot to win a World Championship.

This year, 20 had the second one. Last year, 20 had the fourth one.
The goal is to one day strike lightning on robot design so we can be the sixth one.

In your opinion could a robot provide a skill/ability/trait that by itself would carry the robot to the point where it would win a a world championship?

[Kevin Leonard]

In your opinion could a robot provide a skill/ability/trait that by itself would carry the robot to the point where it would win a a world championship?

71 in 2002. That's it though. Every team since then to win a world championship because of the combination of good strategy, scouting, and a great robot.

And they wouldn't have won had they not correctly identified the right strategic priorities during build season.

[IronicDeadBird]

71 in 2002. That's it though. Every team since then to win a world championship because of the combination of good strategy, scouting, and a great robot.

And they wouldn't have won had they not correctly identified the right strategic priorities during build season.

So do you think theory crafting or the process of looking for game breaking tricks is separate from scouting?

[Kevin Leonard]

So do you think theory crafting or the process of looking for game breaking tricks is separate from scouting?

What 71 did is good strategic design, which is in a similar category as scouting, but not the same.
I don't understand what you mean though.
What you said was:
*INCOMING CONTROVERSIAL STATEMENT*

I'm truly surprised at how intensive some of you guys are with scouting during competition. I mean to be fair this year was the first time I was faced with having to make a pick list. But scouting didn't effect match to match strategy as much as Q'ing up did...

Strategic design isn't scouting, its good design. And they ended up with a good alliance to compliment them (meaning the team that picked them did a great job understanding what they needed to do to be 71's eliminations partner, so they won by doing good strategy and scouting)

[IronicDeadBird]

What 71 did is good strategic design, which is in a similar category as scouting, but not the same.
I don't understand what you mean though.
What you said was:


Strategic design isn't scouting, its good design. And they ended up with a good alliance to compliment them (meaning the team that picked them did a great job understanding what they needed to do to be 71's eliminations partner, so they won by doing good strategy and scouting)

Oh my bad. Generally speaking scouting to me is done as something as soon as I have info on what other teams are doing. So as soon as we get feedback from things like Ri3D or as soon as I start seeing mechanisms crop up on youtube that information is plugged in directly to where its needed, and all that I consider part of scouting. So sometimes the info goes to strategic design, sometimes it goes to manufacturing so they order parts before they all vanish.

[pabeekm]

So do you think theory crafting or the process of looking for game breaking tricks is separate from scouting?

While technically separate processes, scouting/scouting analysis and rules analysis rely on similar skill sets (e.g. knowing how to identify key factors of the game before competition, knowing how certain strategies will likely play against each other given robots of X ability, knowing what qualities of X robot type make or break performance, being able to realistically evaluate your limits (that one especially), etc.). That’s why game analysis and scouting often get paired together within discussions and within teams; the skills that a team uses for one will often be applied for the other. While you could say great scouting is “irrelevant” if you find and realize something truly and absolutely gamebreakinig, the GDC and your competition will try their best to keep you from ever getting there (hence why the few examples are remembered so well).

In the extremely likely event that you can’t totally break the game, even if you have a top tier robot, you are taking on an insane amount of risk by not scouting to your full potential, especially considering how easily various factors not based purely on your ability can murder your end performance (poor schedule, your robot’s name simply not being “out there”, alliance picks going in a crazy direction, etc.); good scouting will enable you to mitigate these risk factors.

That level of risk is way more real than you would think; in 2014 we got hit hard by all of these factors one way or another. The effects would’ve been way worse had we not been scouting hardcore (and 2014 absolutely demanded hardcore scouting), and, in hindsight, there are several ways we could have applied our knowledge to make those factors have hardly any end effect. The main point is that the payoff of great scouting can easily become very high in comparison to the (relatively low) effort cost. That’s why in a competition as intense as FRC, it’s virtually essential for teams who want that extra edge.

[Citrus Dad]

So do you think theory crafting or the process of looking for game breaking tricks is separate from scouting?

We focused from Day 1 this year on can grabbing because we saw it as the chokehold strategy. However we also knew we needed at least a second stacking robot. Then we realized at the end of the build season it would take 2 bots to grab all of the cans, so that became the other scouting priority. So we set out to break the game, but we needed scouting to pull it off.

[IronicDeadBird]

We focused from Day 1 this year on can grabbing because we saw it as the chokehold strategy. However we also knew we needed at least a second stacking robot. Then we realized at the end of the build season it would take 2 bots to grab all of the cans, so that became the other scouting priority. So we set out to break the game, but we needed scouting to pull it off.

We recognized the value of Cans day 1. One day I hope that a team just throws everything out the window and focuses on no direct scoring and pure utility.

Oh and they win worlds... BUt thats impossible in itself.

[evanperryg]

We have the exact same system. Lead scouting mentor and student come down and discuss with our drive team what our alliance should be doing. Time permitting, they come to the pre-match alliance meeting, but are normally quiet unless they drive coach misses something (which isn't uncommon, I miss things :rolleyes: ).
What I wouldn't give to have a scouting mentor... :yikes:

Our system is similar. We wish we had it built by Week 4...:o
It helps when one of your best programmers is spending ~75% of his time working on the app :D Much of our data processing spreadsheets (now replaced by a combination of Access and Tableau, I'm going to post that eventually) were made by just changing some column titles.

Smart strategy and scouting can take a weak robot and get it into eliminations.
Smart strategy and scouting can take a mediocre robot and make it a regional contender.
Smart strategy and scouting can take a decent robot and turn it into a regional lock.
Smart strategy and scouting can take a good robot and make it a sure bet to win a regional.
Smart strategy and scouting can take a great robot and bring it to einstein.
And smart strategy and scouting is essential for an elite robot to win a World Championship.
2nd round picks win worlds...
Nothing on CD has ever been this true.

The definition of pit scouting in my book is asking teams questions about their robot that can't be answered through match scouting. Limit pit scouting to data your actually going to use(anything that can't be answered through match scouting) or in other words like you said, keep it simple(actual questions can be complex, but not redundant to what your match scouters are observing). Don't ask teams questions you can answer through match scouting because It'll be a waste of time and It'll be a pain on Friday night going through redundant pit scouting data.
I'll bite.
Having intensive pit scouting data can make filtering teams by their abilities easier, which is the only reason we still do extensive pit scouting. It has no other use besides giving us criteria to sort quantitative data by. Therefore, most questions we ask are either something they can't lie about (i.e. if they have can pullers, cause I'm standing in front of the robot and I can see if they have them) or something they wouldn't have a reason to lie about (their preferred starting position).


Just about any key data point that you could pick up in the pit--like the ones you mentioned--can also be picked up on the field,
Drivetrain type can be impossible to pick up on from matches, especially if you're trying to differentiate between a really bad swerve and a really good mecanum. Number of motors in drivetrain, another important one when you're looking for defensive bots. Weight, impossible to get from matches. Available autonomous modes, as opposed to your preferred auto mode. Whether a robot can upright a container.

We also ask questions about what the intent of the robot (specifically worded to extract intent of the design). That way, we are able to juxtapose that with how they are doing (and if they are performing to expectations).
I'm not sure how that would give you an idea for how good a team actually is. Some teams have very low expectations, others have unreasonably high expectations.

[Michael Hill]

I'm not sure how that would give you an idea for how good a team actually is. Some teams have very low expectations, others have unreasonably high expectations.

It's less about finding out how good a team is rather than what to expect when we interact with the team. If we know they're going to say they can do [x] during the strategy session, we have data proving whether they can or not (so we know whether to rely on that information). For example, if a team says they're going to put up 3 yellow totes and that they do it all the time, yet our scouting says otherwise, then it's pretty easy to tell them "no" if another team can do it as well. If we see that we are going to be with a coach that's a 10-year veteran mentor, then 9 times out of 10, it's going to be hard to convince him or her of going with a strategy other than their own. It's about arming yourself with information, which I see no problem in.

[EricH]

Drivetrain type can be impossible to pick up on from matches, especially if you're trying to differentiate between a really bad swerve and a really good mecanum. Number of motors in drivetrain, another important one when you're looking for defensive bots. Weight, impossible to get from matches. Available autonomous modes, as opposed to your preferred auto mode. Whether a robot can upright a container.

I suspect that in most years, weight is a non-factor. If you pick them for eliminations, you will naturally do your best to find enough steel plates--or more useful hardware--to bring them up to the maximum weight minus 0.1 lb.

Let me phrase it this way: I don't give a darn about drivetrain type, number of motors, or weight. Why? Because handled well, they. don't. matter. What matters is how you use what you do have. If you have a somewhat lighter 4-CIM 4WD tank, and you drive it effectively, you will do better at your role than a heavy 6-CIM 6WD drop that isn't driven well. This competition (in general) isn't all about the pushing matches--if your 4-CIM hits the corner right when a shot is lined up, they're going to be wasting time realigning while you line up for another shot at them and your partners score 3.


That being said, I think y'all are forgetting something. At the Champs, it isn't just the second pick that will win you the event. It's the third as well. And then the lineups you use.

[Gregor]

Let me phrase it this way: I don't give a darn about drivetrain type, number of motors, or weight. Why? Because handled well, they. don't. matter. What matters is how you use what you do have. If you have a somewhat lighter 4-CIM 4WD tank, and you drive it effectively, you will do better at your role than a heavy 6-CIM 6WD drop that isn't driven well. This competition (in general) isn't all about the pushing matches--if your 4-CIM hits the corner right when a shot is lined up, they're going to be wasting time realigning while you line up for another shot at them and your partners score 3.



When you're picking the 24th robot at a 30-36 team event, drivetrain and number of motors absolutely matters.

That being said, I think y'all are forgetting something. At the Champs, it isn't just the second pick that will win you the event. It's the third as well. And then the lineups you use.

I can't think of a single successful Einstein alliance in the past 2 years that used their third pick for more than one match (although 900 and 5012's counter were certainly interesting, yet ultimately useless).

[EricH]

When you're picking the 24th robot at a 30-36 team event, drivetrain and number of motors absolutely matters.

I maintain that it does not matter. Very simply: If a team can use whatever drivetrain that they have in a manner that benefits the alliance the best, it does not matter how many motors they have, or what type of drivetrain. Plain and simple. I can't say it any simpler. If a 2WD, 2CIM drivetrain is functional and its robot does the job that my alliance needs it to do better than any other robot out there, then darned if I ain't gonna pick that team regardless of however many 6CIM 8WD robots that do that job there are out there.

You cannot change my mind on this.

[Basel A]

I maintain that it does not matter. Very simply: If a team can use whatever drivetrain that they have in a manner that benefits the alliance the best, it does not matter how many motors they have, or what type of drivetrain. Plain and simple. I can't say it any simpler. If a 2WD, 2CIM drivetrain is functional and its robot does the job that my alliance needs it to do better than any other robot out there, then darned if I ain't gonna pick that team regardless of however many 6CIM 8WD robots that do that job there are out there.

You cannot change my mind on this.

Yes, and if a robot is 90% composed of cheese does the job that my alliance needs it to do better than any other robot out there, then I'll pick it. But the fact of the matter is that a robot 90% composed of cheese is unlikely to do any job well, particularly not the jobs often assigned to 3rd partners like "getting in the way of fast robots." A little pit scouting could save you time there.

[EricH]

But the fact of the matter is that a robot 90% composed of cheese is unlikely to do any job well, particularly not the jobs often assigned to 3rd partners like "getting in the way of fast robots." A little pit scouting could save you time there.Or, it could make you write off a team as useless. If that 90% cheese robot manages to shut you down, particularly if you wrote it off due to your pit scouting, I'd suspect that you'd have a little whey on your face.

[gblake]

... You cannot change my mind on this. Ungracious Iconoclast Alert!!!! ;)

You would be telling oh-so-many people that they have wasted oh-so-much of their own time, and oh-so-much time of the people they were interviewing.

You can't do that without expecting a full pitchforks & torches parade in your honor.

But what if it really was true??? (or 80% true?)

Empires would fall. Seas would boil. Cities would crumble and be swept into the sea. Flocks of black helicopters would emerge from Hollow Earth. Cats and dogs would start getting along. ...

And many teams could invest more of their students' talents, and precious time, in something that would pay greater dividends?

Blake

[jajabinx124]

I can't think of a single successful Einstein alliance in the past 2 years that used their third pick for more than one match (although 900 and 5012's counter were certainly interesting, yet ultimately useless).

I think 3rd picks at champs can be crucial for a successful Einstein alliance. Take a look at TBA and you'll see that. Take the Hopper division winners and Einstein finalists for example(successful in my book). 2512 was a third pick and yet they were crucial for using 987's cheesecake to grab cans from the center and utilizing the other HP station to make stacks.

Many alliances won their division because of 3rd picks as well this year. Take the carson division winners as an example. 1711 was crucial to their Einstein appearance this year by grabbing cans from the center. Tesla division winners as well with 2526 as their 3rd pick.

[evanperryg]

Or, it could make you write off a team as useless. If that 90% cheese robot manages to shut you down, particularly if you wrote it off due to your pit scouting, I'd suspect that you'd have a little whey on your face.
Except if you're doing good scouting, this doesn't happen. It's also why you don't eliminate teams solely based on their pit scouting data.

I suspect that in most years, weight is a non-factor. If you pick them for eliminations, you will naturally do your best to find enough steel plates--or more useful hardware--to bring them up to the maximum weight minus 0.1 lb.
Let me phrase it this way: I don't give a darn about drivetrain type, number of motors, or weight. Why? Because handled well, they. don't. matter. What matters is how you use what you do have.
So what if it doesn't matter most years? Weight mattered this year.

It's a given that your performance is based on how you use your robot, but having what would be most productive for the alliance is also important. I've seen a rookie team with a 2CIM drivetrain picked by the first seed alliance because their driving was that good. In 2014 we picked 2 partners both with mecanum, who both had some amount of a defensive role. Sounds like a terrible idea, I know, but as the 8th seed alliance captain up against the two top robots at the event, we knew we have to try something daring if we even wanted a chance. We knew both of these teams could use their tall robots and mecanum drives to get in front of the much shorter high-scoring robot we would face. Instead of pushing, they just stayed in front of the opposing shooter. We were surprisingly effective, considering we were the 8th seed alliance on a field that was basically 5 teams deep. How these teams used their robots was very effective, but the fact that we knew what they actually had allowed us to better compare them to other robots with similar defensive abilities, who would not have performed as well because of what they had. Sure, the "how" is vital, but that's picked up in qualitative data reliably. Pit scouting is about the what- the basic criteria. It isn't about deciding who's better, it's about being able to easily sort out robots by their physical features.

If you have a somewhat lighter 4-CIM 4WD tank, and you drive it effectively, you will do better at your role than a heavy 6-CIM 6WD drop that isn't driven well. This competition (in general) isn't all about the pushing matches--if your 4-CIM hits the corner right when a shot is lined up, they're going to be wasting time realigning while you line up for another shot at them and your partners score 3
Obviously, our questions vary with the context of the game. Obviously, asking how many motors are on your drivetrain was not relevant to RR, and it was not something we asked this year. However, in 2014 it would have been important when looking for a pusher.


That being said, I think y'all are forgetting something. At the Champs, it isn't just the second pick that will win you the event. It's the third as well. And then the lineups you use.

Believe me, I know. 1089 got us our only step can through all of Einstein semis. (also, thanks to 548 for the cheesecake)

Long story short, pit scouting is all about the what. It's just a simple little criteria that can be used to sort teams by their physical attributes, not by how well they do it. The how is determined by qualitative and quantitative match data. Pit scouting questions are adapted to the context of the game, for obvious reasons.

[evanperryg]

Also, more to the topic of this thread, Amit3339 if you'd like more detail on any aspect of how our scouting team is managed, feel free to PM me.

[Michael Corsetto]

I maintain that it does not matter. Very simply: If a team can use whatever drivetrain that they have in a manner that benefits the alliance the best, it does not matter how many motors they have, or what type of drivetrain. Plain and simple. I can't say it any simpler. If a 2WD, 2CIM drivetrain is functional and its robot does the job that my alliance needs it to do better than any other robot out there, then darned if I ain't gonna pick that team regardless of however many 6CIM 8WD robots that do that job there are out there.

You cannot change my mind on this.

In an ideal world, with perfect scouting, strategy and foresight, I'm sure this holds true. Makes sense you don't want to deviate from this theory.

However, I want to emphasize that, in practice, filtering pick lists based off of pit data can be very helpful. I call it "strategic generalization".

For 2010-2014 era games, 1678 found that making an initial filter based on drivetrain type improved the quality of our 2nd pick. I think this is what Gregor was talking about.

2015 was a real outlier in terms of 2nd picks. We used no match scouting data for all three of our regional 2nd picks, only cheesecake pit scouting data. We initially filtered teams based off of code type, weight, willingness to work with us, etc., all data collected in the pit.

Lesson learned from 2015, past generalizations will not always apply to future games. The key is not "will you generalize?" but "what will you generalize?"

I won't try to change your mind, but hopefully others find some value in the practice of strategic generalization.

You can hold on to your ideas and I'll hold on to my blue banners.

-Mike

[Citrus Dad]

Well that and caffeine...

Delivered using multiple pathways.;)

[Citrus Dad]

We recognized the value of Cans day 1. One day I hope that a team just throws everything out the window and focuses on no direct scoring and pure utility.

Oh and they win worlds... BUt thats impossible in itself.

Last year, despite being the alliance captain, I believe that we did not score a single goal during the Champs playoffs. We did score truss points as that was our primary design role. I think 1640 scored more than us (1 goal on Einstein.) We scored 232 in a regionals qual match where we scored only 1 point goals.

[gblake]

...
For 2010-2014 era games, 1678 found that making an initial filter based on drivetrain type improved the quality of our 2nd pick. I think this is what Gregor was talking about.Do you suppose that it's possible that an even more effective and useful filtering or ranking could have been done using accurate observations of how the drivetrains (and drivers) performed on the field, instead of using the results of your pit scouting?

I'll wager a nice dinner that it is more than just possible; and I think that is what EricH was talking about.

I'll let you guys debate which methods get you to your desired end goals at the lowest costs (in terms of team resources expended), and in the time frames when initial and final estimates are required.

... You can hold on to your ideas and I'll hold on to my blue banners. In my best Jar-Jar Binks voice, "How wude!" ;)

That last line came across a little harshly, especially when earning your blue banners is affected by much more than just scouting accuracy. Good enough scouting is important, and pretty close to being necessary; but it is way far away from being sufficient.

Blake

[Michael Corsetto]

Do you suppose that it's possible that an even more effective and useful filtering or ranking could have been done using accurate observations of how the drivetrains (and drivers) performed on the field, instead of using the results of your pit scouting?

Can the answer be both? Notice I said our "initial" filtering was based off drivetrain type aka. pit scouting. Our second filter was based on "accurate observations of how the drivetrains (and drivers) performed on the field." The primary filter made our second filter easier to perform.

Richard can talk more about how we implemented our second filter based on field performance, I don't understand half of how they do it! :ahh:

In my best Jar-Jar Binks voice, "How wude!" ;)

That last line came across a little harshly, especially when earning your blue banners is affected by much more than just scouting accuracy. Good enough scouting is important, and pretty close to being necessary; but it is way far away from being sufficient.

Definitely agree, winning in FRC is a many-dimensional challenge.

On topics like scouting, with a wide range of opinions, voices and methods, we prefer to "steal from the best, invent the rest." We follow that motto in all aspects of our team's operations, and it's definitely worked out for us. YBBMV (Your Blue Banners May Vary)

-Mike

PS. For 1678, "the best" almost always refers to 254 or 973. We copy them a lot. (http://www.citruscircuits.org/wp-content/uploads/2014/01/Madtown-8.jpg)

[RoboChair]

Do you suppose that it's possible that an even more effective and useful filtering or ranking could have been done using accurate observations of how the drivetrains (and drivers) performed on the field, instead of using the results of your pit scouting?

No, because we do both. We pre-filter based on what their drivetrain is and weight it with their on field performance. It is never black and white, but many different shades on a grey scale when it comes to us comparing robots. Though some of those shades can be practically indistinguishable from black and white, they are in fact at least a little grey.

Edit: Dang Mike, you post sniped me!

[Siri]

That last line came across a little harshly, especially when earning your blue banners is affected by much more than just scouting accuracy. Good enough scouting is important, and pretty close to being necessary; but it is way far away from being sufficient.Without commenting on verbiage, having had first-hand experience with a 1678 blue banner I can say that it absolutely is about scouting. Not that it isn't also about other things, but I'm utterly positive that these nice folks would would have fewer medals (and so would we!) if not for their incredible scouting work. It's not sufficient, but it is absolutely necessary the way they play. I suspect that Mike was implying necessity rather than sufficiency with that comment.

[gblake]

Can the answer be both? Notice I said our "initial" filtering was based off drivetrain type aka. pit scouting. Our second filter was based on "accurate observations of how the drivetrains (and drivers) performed on the field." The primary filter made our second filter easier to perform.

Richard can talk more about how we implemented our second filter based on field performance, I don't understand half of how they do it! :ahh: Right, and that is our violent agreement about me stepping aside if folks want to discuss whether the resources consumed in their own team's pit scouting is worth trouble.

Spending a little time and having the downstream work get a lot easier means it is worth the trouble. Spending a lot of time and only getting a minor simplification later would not be worth the trouble. I don't think outside observers can make useful generalizations about that, for other teams. There are a lot of implementation details in play.


Definitely agree, winning in FRC is a many-dimensional challenge. A little more violent agreement.

No, because we do both. We pre-filter based on what their drivetrain is and weight it with their on field performance. It is never black and white, but many different shades on a grey scale when it comes to us comparing robots. Though some of those shades can be practically indistinguishable from black and white, they are in fact at least a little grey.What you wrote here doesn't support your answer of "No".

If a team is prefiltered to the bottom of the heap because of robot implementation, and does well on the field, do you keep them ranked lower than teams that do worse than them on the field?

If a team is prefiltered to the top of the heap because of robot implementation, and does poorly on the field, do you keep them ranked higher than teams that do better than them on the field?

My belief is that as the quals draw to a close, accurately assessing on-field performance, plus making a small investment in the pits near the end of quals (to find out if a team finally starts hitting on all cylinders (they fix a software bug, or complete a mechanical change, or swap drivers, or ...)), swamps any investment in early pit scouting.

I think Michael wrote that pit scouting helps you a bit with deciding who gets the most scouting attention while on the field. That makes sense.

However, it sounds like you are trying to tell me that in your method, at the end of quals, if you ranked the teams according to the on-field performance your scouts see, you would then also adjust those ranks non-trivially based on pit-scouting data. That sounds a bit odd. I can certainly see making a case for it because the number of qual matches played usually isn't enough to supply an excellent assessment of each teams abilities. But ... with that in mind, I think we might at least agree that as the number of qual matches increases (and for the sake of argument, lets assume everything else is constant), the value of pit-scouting data steadily declines.

What I was saying in support of what I think EricH was saying, is that by the end of a typical tournament, I would side with him and be unlikely to let early pit-scouting data significantly alter any ranking I had created using on-the-field scouting.

If you guys do let pit-scouting data significantly affect your end-of-quals rankings I'm surprised. And, if you do, maybe that has helped you win, or maybe you have won regardless of any possible harm done by those changes. Get out a ouija to answer that one.

Regardless, congrats on the wins.
Blake
PS: In all of this I am setting aside aspects of team performance that depend on how well any two teams get along when they need to communicate/cooperate. For the sake this discussion, let's assume everyone is equal in that regard, and in other similar characteristics.

[EricH]

... I'll hold on to my blue banners.

From my high school years on 330, doing scouting, that'd be 9 gold medals in my room (including 2x RCA), plus a trio of silvers. Just so you have a known point of comparison on that.

[Edit]: I did go and check on the banners Mike has won since high school. If high school were the only factor, I think I've got that hands down. But, if we're dealing with total, or current teams, he's got me by at least 3 depending on exact method used.

[RoboChair]

The reason that I said no in the first place is that we cannot make a fully informed decision without both. Every year the weights will change but we will always need both sets of data by the time all is said and done. We cannot make an fully informed decision with out a full set of information. It would be incredibly rare for a robot's drivetrain to change our choice of 1st pick, but by the time it comes back around for the 2nd pick things are rarely simple. The drivetrain a robot is running may drastically change our strategy as an alliance based solely on what it is best and worst at doing. Some years our pick list has contingencies, while others have contingencies within contingencies.

[evanperryg]

However, I want to emphasize that, in practice, filtering pick lists based off of pit data can be very helpful. I call it "strategic generalization".

For 2010-2014 era games, 1678 found that making an initial filter based on drivetrain type improved the quality of our 2nd pick. I think this is what Gregor was talking about.


A better-said version of what I was trying to say. Although it isn't usually helpful for much else, pit scouting data can make a good filter.

Do you suppose that it's possible that an even more effective and useful filtering or ranking could have been done using accurate observations of how the drivetrains (and drivers) performed on the field, instead of using the results of your pit scouting?
I'll wager a nice dinner that it is more than just possible; and I think that is what EricH was talking about.


I'm sure it's possible, but it would be highly subjective. I'm not saying that subjective information is bad, but trying to use subjective information to filter quantitative data is unreliable, at best. Say you ranked each team's drivetrain/driving abilities on a scale of 1-5, and only viewed the 4s and 5s. This, of course, raises a few inevitable questions:
-what's the difference between a 4 and a 5? You'll need a list of differences between a 4 and a 5, which will include even more subjective criteria.
-how important is the difference between a 4 and a 5, versus the more objective quantitative data? What if there's a team with a 5 that can't do any scoring, but there's a 4 that could score a few points in auto, and a few in teleop if needed?
It's probably possible, but it's the kind of thing that would get very messy very easily. It's better to do your first-order sort by easily-quantified information, then take into account more subjective information to do more detailed sorting. It makes the entire picklisting process more efficient.

[Kevin Leonard]

/snip/
Say you ranked each team's drivetrain/driving abilities on a scale of 1-5, and only viewed the 4s and 5s. This, of course, raises a few inevitable questions:
-what's the difference between a 4 and a 5? You'll need a list of differences between a 4 and a 5, which will include even more subjective criteria.
-how important is the difference between a 4 and a 5, versus the more objective quantitative data? What if there's a team with a 5 that can't do any scoring, but there's a 4 that could score a few points in auto, and a few in teleop if needed?


Oh god subjective rating systems make me cringe.
For some reason, in 2013, we had a metric on some of our match scouting sheets called "Speed". It was a rating of 1-5 based on what they saw during the match. We never used it for strategy (so I don't know why we had it), but it was funny to see what different students rated different robots. Sometimes teams with 2-speed, aggressively geared drivetrains were given 1's and 2's, while some robots with single-speed, relatively slow drivetrains were given 4's and 5's. Most notably was the fact that somehow our single speed 12 fps tank drive from that year had the highest "average speed rating" at the event, due to obvious bias in the scouts.

We've tried since then to weed out poor, subjective rating systems like that.

[gblake]

I'm sure it's possible, but it would be highly subjective.If I knew a way to write this without having come across snarky, I would do it, but I not that good of a wordsmith today: Thanks for making me smile.

I understand what you are saying. I suppose I just reach a different conclusion when I think about the minimal set of observable variables I would use at the end of quals.

STEM robotics has plenty of room for both approaches.

Blake

[Citrus Dad]

Oh god subjective rating systems make me cringe.
For some reason, in 2013, we had a metric on some of our match scouting sheets called "Speed". It was a rating of 1-5 based on what they saw during the match. We never used it for strategy (so I don't know why we had it), but it was funny to see what different students rated different robots. Sometimes teams with 2-speed, aggressively geared drivetrains were given 1's and 2's, while some robots with single-speed, relatively slow drivetrains were given 4's and 5's. Most notably was the fact that somehow our single speed 12 fps tank drive from that year had the highest "average speed rating" at the event, due to obvious bias in the scouts.

We've tried since then to weed out poor, subjective rating systems like that.

Our solution is to rank robots within their alliance in an ordinal fashion (although there can be some cardinal ranking, e.g., no one is worth a 3 as the best), and then to pick the best of the match in each dimension, giving them a 4. We worked through the math and assuming the transitive property, the distribution of rankings fall quite close to a cardinal ranking system that relies on our "superscouts" keeping a constant metric across the entire tournament. The system worked extremely well in 2014. (As Mike has said, this system wasn't very important in this year's game.) We're looking at adjusting that ranking system to use the variance of the scoring distribution to standardize the metric.

I think this is where I take over from Mike. The next twist is that we use our quantitative scouting system results and the match scores from the previous competitoin to run predicted scores. We then add in our qualitative scores as defensive effects and minimize the squared error using Solver by varying the weights of those qualitative scores. We're then able estimate the defensive contribution expected for a given qualitative score and the relative weights for each dimension. For example, I think we found the 4814 contributed about 20 points a match (maybe higher?) in defense in the 2013 Curie Division which was multiples of the next robot.

However, it sounds like you are trying to tell me that in your method, at the end of quals, if you ranked the teams according to the on-field performance your scouts see, you would then also adjust those ranks non-trivially based on pit-scouting data. That sounds a bit odd. I can certainly see making a case for it because the number of qual matches played usually isn't enough to supply an excellent assessment of each teams abilities. But ... with that in mind, I think we might at least agree that as the number of qual matches increases (and for the sake of argument, lets assume everything else is constant), the value of pit-scouting data steadily declines.

What I was saying in support of what I think EricH was saying, is that by the end of a typical tournament, I would side with him and be unlikely to let early pit-scouting data significantly alter any ranking I had created using on-the-field scouting.

If you guys do let pit-scouting data significantly affect your end-of-quals rankings I'm surprised. And, if you do, maybe that has helped you win, or maybe you have won regardless of any possible harm done by those changes. Get out a ouija to answer that one.

Regardless, congrats on the wins.
Blake
PS: In all of this I am setting aside aspects of team performance that depend on how well any two teams get along when they need to communicate/cooperate. For the sake this discussion, let's assume everyone is equal in that regard, and in other similar characteristics.

We don't use early pit scouting data other than to get pictures, and probably the drive train configuration. In fact as the season goes on we use previous competition results to pre-seed our scouting data, and we progressively replace that pre-competition data with actual matches. We run a regression of our scouting data on the OPR metrics to estimate the relationship to our quantitative scouting parameters.

We get pit scouting and drive team information as the competition goes on. We've had specific task questions the last two years about robot configuration that we can't really see from the stands, and that our scouts probably can't discern. Our drive team and match tactician gives input about working with particular teams.

We do the quantitative ranking and then we use the pit scout and drive team info to move teams up and down. The fact is that 10-12 matches is not enough observations, and those observations are not independent of each other. Teams change performance over the tournament. The initial ranking is a starting point. Then we introduce the non quantifiable factors such as drive train configuration (no mecanum until this year), robot configuration and team cooperation. And we include our past experiences. We moved both 1671 and 5012 up our list because of positive experiences with their organizations.

So in the end, it may not be pit scouting that trumps our initial rankings, but it is qualitative assessments that are not feasible by our field scouts.

[evanperryg]

We get pit scouting and drive team information as the competition goes on. We've had specific task questions the last two years about robot configuration that we can't really see from the stands, and that our scouts probably can't discern. Our drive team and match tactician gives input about working with particular teams.

We do the quantitative ranking and then we use the pit scout and drive team info to move teams up and down. The fact is that 10-12 matches is not enough observations, and those observations are not independent of each other. Teams change performance over the tournament. The initial ranking is a starting point. Then we introduce the non quantifiable factors such as drive train configuration (no mecanum until this year), robot configuration and team cooperation. And we include our past experiences. We moved both 1671 and 5012 up our list because of positive experiences with their organizations.


This is almost exactly what we do, except our pit scouting question list is generally very long and uses language that approaches legalese. This will be changing in the coming season as we've found that a lot of the criteria go unused. Our pre-scouting data is largely qualitative but we have a spreadsheet that runs through a team's season info and shows what events a team was at, component OPRs, etc. The most valuable pe-event data we get is notes from scouters watching match videos. Early in the event, we rely on those notes heavily in doing match strategy, and we slowly progress over to using event data. Our picklists are primarily based on quantitative performance in relation to what type of robot we want. (i.e. the first picklist this year started with the highest-scoring feeder bot with a stack auto) Second picks are much more based in utility, not in scoring ability. Teams move up, down, or off the list based on qualitative data, our personal experiences with them, and drive team comments. Some teams are immediately put on the DNP list because of repeated bad experiences with them, although this is uncommon.

[Edxu]

Our strategy sub-team is pretty small (4 members, 1 leaving this year), but we play a great role at competition in terms of communicating between teams and planning matches.
We've recently converted to a tablet scouting system, which has the benefit of giving us real-time data on the performance of a team at a competition, allowing us to plan matches better, using hard data as the base.

To answer OP's questions:

-Strategy Brainstorming
At the start of every build season, 610 watches the game release and then we break for approximately 2 hours while each team member reads the game rules. In that time, a collaboration between the Strategy and Administration heads create a rules quiz that each member must pass in order to participate in the build process.

People who pass are allowed to participate in the Textbook Strategy discussion, which is basically a round-table discussion between the entire team, led by the Strategy division, where ideas are thrown around and considered. By doing this as a team, we can get on the spot estimates of whether or not something is feasible, as well as getting the attitude of the team on a certain idea. For example, this build season, there were a few people who were initially opposed to our idea of being a container specialist without the ability to handle totes, but by calculating the maximum possible score with containers won on our side, the Strategy division showed that the middle containers were incredibly important to any Einstein-winning alliance.

The Strategy should drive the Mechanism, not the other way around. By designing a component that specifically fulfills a task, it's more likely to be successful, and it also gives your Mechanical guys a goal to work towards (eg this year: We need a 4-bin mechanism).

In terms of Strategy's purpose at competitions:
-Plan matches with Alliance Partners using scouting data off tablets
-Organize expert scouts and create a list of special traits to watch out for (This year again: opposing canburglars, strong stacking robots that synergize and robots that may have the potential to add canburglars for Playoffs).
-Explain match strategy to the Drive Team, making sure that they know where our Alliance Partners will be throughout the match and what they'll be doing. This makes it much easier for our Drive Team to focus on what they need to do without worrying about the rest of our Alliance.
-Walk around the pits and act as "superscouts" who look for tiny traits that may be useful in an alliance partner.

If you have any other questions, feel free to shoot me a message.

[Citrus Dad]

Our strategy sub-team is pretty small (4 members, 1 leaving this year), but we play a great role at competition in terms of communicating between teams and planning matches.
We've recently converted to a tablet scouting system, which has the benefit of giving us real-time data on the performance of a team at a competition, allowing us to plan matches better, using hard data as the base.

To answer OP's questions:

-Strategy Brainstorming
At the start of every build season, 610 watches the game release and then we break for approximately 2 hours while each team member reads the game rules. In that time, a collaboration between the Strategy and Administration heads create a rules quiz that each member must pass in order to participate in the build process.

People who pass are allowed to participate in the Textbook Strategy discussion, which is basically a round-table discussion between the entire team, led by the Strategy division, where ideas are thrown around and considered. By doing this as a team, we can get on the spot estimates of whether or not something is feasible, as well as getting the attitude of the team on a certain idea. For example, this build season, there were a few people who were initially opposed to our idea of being a container specialist without the ability to handle totes, but by calculating the maximum possible score with containers won on our side, the Strategy division showed that the middle containers were incredibly important to any Einstein-winning alliance.

The Strategy should drive the Mechanism, not the other way around. By designing a component that specifically fulfills a task, it's more likely to be successful, and it also gives your Mechanical guys a goal to work towards (eg this year: We need a 4-bin mechanism).

In terms of Strategy's purpose at competitions:
-Plan matches with Alliance Partners using scouting data off tablets
-Organize expert scouts and create a list of special traits to watch out for (This year again: opposing canburglars, strong stacking robots that synergize and robots that may have the potential to add canburglars for Playoffs).
-Explain match strategy to the Drive Team, making sure that they know where our Alliance Partners will be throughout the match and what they'll be doing. This makes it much easier for our Drive Team to focus on what they need to do without worrying about the rest of our Alliance.
-Walk around the pits and act as "superscouts" who look for tiny traits that may be useful in an alliance partner.

If you have any other questions, feel free to shoot me a message.

Ditto! How did you explain what we do so much more elegantly than me? :o

[Citrus Dad]

We have two presentations (slides + video) on our season strategy and scouting strategy. We are trying to get them up on our website.

As promised (http://www.chiefdelphi.com/media/papers/3153).