![]() |
Understaffed scouting
We have a relatively small team but would like to have an effective scouting group. How might we scout matches effectively with fewer than six people? I'm not worried about pit scouting, as that can be done by one or two people.
--josephus |
Re: Understaffed scouting
Quote:
Or partner with another team who does have such a system for an event or two. You get the data and learn one way to scout. If both of those ideas fail... I would suggest running a 2-person team, with a backup 2-person team at any given time. Each person scouts one side of the field; the backups double-check. |
Re: Understaffed scouting
Quote:
|
Re: Understaffed scouting
If nothing above works, then you could consider this reduction. I've never tried it, and I kind of just dreamed it up right now, but it might put you on the right track. It technically requires two people for the simplest form, but gets worthwhile with three.
Take a look at the scouting sheets being published for a single team and adapt them to a whole alliance. # of totes an alliance stacks, autonomous points for the whole alliance, total points of the whole alliance, litter thrown, etc. Then, you'd plug the data collected for one alliance into each robot on the alliance's individual sheet (in your data collection program), and get a very rough approximation. It's basically the same concept as OPR, but without calculation and being more specific than straight points. You wouldn't even need to make a system different from one for single robots, the data would just be bigger. You could see a general trend that "Oh, when team yyyy is on the field, the highest bin scored is generally higher." Like OPR, that trend might be totally off base, but you could see a trend and corroborate it with a "common sense check". If you're not doing data entry, the drive team needs to come up to the stands and get the data themselves. You'll have to just stick everything in a binder, put the match schedule on front, and hope they can find what they're looking for. I can see some terrible times trying to deal with that at competition*. In other words, if you intend to do it this way, it looks like data entry will be almost essential. * "Hey coach, let's go up to the binder and look through 100 matches to find some numbers on our two alliance partners for this last qualification match!" = Not Fun. |
Re: Understaffed scouting
If you are seriously screwed and in a position with almost no members, you could have two people... One per side.
The kicker would be, No defensive Rating :))))) |
Re: Understaffed scouting
I've heard from lots of NC teams that they're in the same boat as you guys; they want to scout but would need to team up with someone to have enough people. I would suggest shooting Marie Hopper an email and asking her to send out to all NC teams asking if anyone wants to share scouts. There are likely several teams who would be interested. Good Luck!
|
Re: Understaffed scouting
Last year what we did is we paired up on scouting and worked together with other teams. IT was a really unique experience and were able to use less members but still get valuable data from it.
|
Re: Understaffed scouting
I would actually do something a little different than some of the suggestions already posted. If I only had only around three scouts, I wouldn't focus too much gathering data. I think scouting an entire alliance will lead to missing too much data (bad/incomplete data isn't very helpful) and scouting one robot per match will lead to extremely small sample sizes. I would rather have subjective data from several matches than knowing a team stacked 5 totes in the one match and no idea what they did in the other five.
Instead, I would focus on making sure watch your upcoming alliance partners (and opponents in other year) in their couple of matches before you are with/against them. Try to become experts on these teams first. Still be aware of what some of the other teams are doing, but your main goal in qualifications should be to win every match (or have the highest average). Then, if it is looking like you may finish the in top 8 (or be picked early), then start really scouting teams that compliment your strategy. On the whole, I think too many FIRST teams put too much focus on gathering, organizing and displaying data and not as much as actually using the data and learning how robot gets those stats. Way too often I talk with other scouts that can read a report about a robots average in ten different stats, but can't tell you one thing about the actual robot. It is pretty amazing how much credibility you gain in pre-match strategy if you come up to a team and say "I like what you did your last match doing xyz. You scored x points, right?". So, in this respect, not having a huge scouting team is not as detrimental as it may seem. |
Re: Understaffed scouting
Inter-team scouting efforts can be not only a fantastic tool for gathering data, but also a great way to make new friends on other teams.
For the past few years, Team 20 has run our scouting program, The Constellation, with a number of teams at each of our events. In 2013, we had the opportunity to scout with Team 4265, the Wildbots, at both Archimedes and IRI. Since then, 4265 has become one of our team's best friends in FRC, despite the fact that they live quite a large distance away from us. If no teams in your area run a similar program, find another team in a similar situation as you and pair up with them for scouting. You could also go to a more experienced team in your area with an established scouting program and ask them for help. Most teams would be willing to give it a shot. |
Google spreadsheets are always the best.
|
Re: Understaffed scouting
What we have done in past years is a lot of gut feelings. When watching a match, what you see, whether in the stands or in queue van make a big difference in who you would choose. Trust your gut, and hope it the 10%of the time it is wrong isn't then. :)
|
Re: Understaffed scouting
Quote:
That being said, I support the group scouting method where you work with other teams. Its a little harder to guarantee quality data, but its far superior to no data at all. |
Re: Understaffed scouting
Inter-team scouting is a good way of doing it, if both teams arrange it beforehand. (My one caution would be to have people from the other team scout your robot to reduce bias).
If you have <=6 people: Have two of them scout each alliance as a whole (swap them in and out as need), and someone compile the data between matches. Then have the remainder scout individual robots determined at the start of the match for specific qualities. In the early matches, pick teams that are likely to be captains or first picks so you can sort out the top. As matches progress, start going with teams who could make second picks, but still check the top teams. Create a sketch of a list at lunch, see if you can compare teams close to each other in close matches (if team A, B, and C are all next to each other in the list and all play in four matches, have one guy watch them during their respective matches, then do a sub sort). A number of teams already do the above, in addition to the 6+data compiler scouting teams. The only thing if you do this is that the 2-5 people should be the same 2-5 people all day, not 2-5 who switch off with pit crew*. Like designing robots, it's better to have a few experienced scouts than a lot of okay scouts. If you have the same few scouts running the team the whole season, you will all get better at it by the time champs rolls around. *If pit crew has input about how certain teams work with your team or suspect failure points in other machines, take it into consideration. |
Re: Understaffed scouting
Quote:
|
Re: Understaffed scouting
Quote:
Regarding the 1114/2056/4334 alliance (warning: *some* blind conjecture ahead) yes, they were well driven, dependable and thin, but there are elements to consider that can't be answered without numbers. 1114 and 2056 would be taking a gamble by having only 2 robots that could shoot when facing alliances of 3 if they didn't do their homework. With good scouting data, you can look at how much higher robots scored compared to their averages when they had 4334 feeding them. If shooters score an average of 2 more cycles when they have 4334, then it would be worth choosing 4334 over any robot that scores an average of 2 cycles or fewer. By self scouting, they can also put together hypothetical alliances and estimate not only their expected match score, but also the standard deviation of scores of different alliances (high standard deviation = higher risk/higher score ceiling). Lacking this information means you can't make decisions about things like your ceiling vs your average, or what your opponents are likely to be scoring. This is a very condensed version, but there is a lot more that can be gleaned from good scouting data than what I said here. I have never won a world championship though, so if anything I put up is wrong and can be corrected by someone who has, I welcome it. :) |
Re: Understaffed scouting
I think there's a lot of conflation of ideas going on here. What you do about scouting when you're resource-limited depends almost entirely on what you're trying to achieve. (Much like everything in life.) Are you legitimately aiming to be an alliance captain and make a pick list that can reasonably win you this competition? Then your best and only reliable solution is probably to not be understaffed (or under-trained, for that matter). Realize that this is a two step scouting issue: become an alliance captain and make your pick list.
But scouting that way when it's not your main/achievable goal that weekend won't necessarily help you. If your goal is to play well and get picked, 'gut' scouting isn't so bad. A few well-trained scouts/strategists can give you a lot of insight into your allies and opponents without actually tallying game pieces. In fact, when your team is understaffed but not under-trained in this way, qualitative can be better than quantitative. I've always opted for good qualitative over bad quantitative, and it hasn't let me down yet. On the third hand, if you're aiming for either of these two and/or to get deeper into the FRC community, joint-team scouting can be great. Just understand it has its pitfalls. On the fourth hand, if you goal is just to get better at the game (including scouting), you probably want a mix of qualitative and quantitative for your own team. I and most of the coaches I play with will keep at least one top scout on 'gut' duty--usually more than one--whenever we're fully staffed. That means at some point those guys need gut scout training. On the other hand, good quantitative scouting also requires practice. Much like everything in this business, your scouting strategy needs match your competition strategy. Be honest with yourself - don't discount a strategy that (probably) won't make you Championship Alliance Captain when that's not your team's goal at the given moment. |
Re: Understaffed scouting
what do you mean by scouting apps? Has someone developed an application for scouting matches?
|
Re: Understaffed scouting
Quote:
The problem with scouting under the assumption you won't be an alliance captain is that sometimes you still end up an alliance captain. Every team should have a picklist going into Saturday morning, because a) they could end up an alliance captain, and b) they could get picked by a team with no pick list. In my opinion and experience, quantitative scouting is almost always better than qualitative. That being said, if you're unable to put together the people in your team or through multiple teams to have a quantitative scouting group, qualitative is better than nothing (and is often a good supplement to hard data anyway.) |
Re: Understaffed scouting
Siri put it right, whereas I lost track of the problem statement. How you scout should depend on your goals. In my time as a scout, the goal was to make a pick list, but if you are looking to get picked, a better goal may be to scout weaknesses in potential opponents (ex: finding what 2013 cyclers could not drive through the pyramid so you know how to play defence on them).
If you are in picking position however, I would be wary of using your gut for anything more than choosing between two candidates who are worth the same amount of points on paper, but maybe play differently. |
Re: Understaffed scouting
Quote:
But pick list strategy and scouting strategy are not necessarily the same. In the best case, you want them to be the same, but if that's not realistic it doesn't necessarily make sense to conflate them: Pick List: Every team from 1st to nth should write a pick list: you might have to use it, and it's a necessary skill to acquire should you wish to improve. But make no mistake, a winning pick strategy is very, very difficult to make (speaking as someone who's done it both right and wrong before). Like everything, Captaining takes practice. My Point: What my argument does mean is you shouldn't be surprised your list isn't as good as one backed by an experienced, comprehensive scouting system. It can't be. Your [used as a general pronoun] list would not have been as good as theirs even if you tried to mimic their system, because you're just not ready for it. In fact, it would almost certainly be much worse that you actual list. Because there is such as thing a bad/badly used quantitative data, and it shows up a lot in untrained scouting systems, whether or not it's identifiable at the time. So do what you're most capable of that's most (likely to be) useful to you. Perhaps more importantly, remember that being picked and forming part of an alliance that you want--particularly at upper-tier events--is in fact an active and difficult job. It takes energy and practice, and you need to decide how to allocate those resources. Having been both there and in the "Hey, you're an alliance captain!" spot, I still won't let the fear of Captaining prematurely interfere with giving the team its best possible shot at peak performance. 1640's system is getting better at Captaining, but I've also had great 'gut' scouts that help me play our best in quals while "selling" the team for different alliance strategies. That's not to say that qualitative is necessarily best when you're understaffed, or that quantitative doesn't help you with qual strategy. You need to have people who are truly good at either for it to work. We've evolved through approaches that fit our skill sets and situations at the time. (Notably Einstein Finalists 2014, Einstein Semifinalists 2013, MAR Champions 2013, MAR Champions 2012. We also won Philly 2011 as 2nd Captain.) Don't mimic, emulate. On a separate note, I do coach and select from the "gut" (or just not purely quantitatively) in other situations. Picking needs quantitative data, but there's a lot more to it than that: how do we play together, how do we work together, how do you think their crew will handle the pressure, and so on. That discussion might make for another interesting thread. |
Re: Understaffed scouting
I would generally regard "gut" data as being a tiebreaker at best. Something like "Data says these two are about the same, which do I think will work better?"
|
Re: Understaffed scouting
Quote:
Once you figured that out, then focus your scouting on the information you need. Try to keep it from being overwhelming for your scouts. And as EricH said, pull in mentors and parents. We've done that in the past, even having junior mentors testing out specialized scouting skills, and I've sat in for students when they need a break. Really make it a team effort. |
Re: Understaffed scouting
First to cover a few things on this thread. Using Google spreadsheets while not at an event could be a useful tool; however, since most events will not allow you to run your own WIFI, using Google spreadsheets is not a viable option while in the stands. Secondly for the person that asked, yes Scouting apps, many teams develop their own or use ones published by other teams. There is a whole thread on this topic Scouting the Scouting apps.
Secondly what has been mostly ignored is that in order to get the best data possible there needs to be a mix of both qualitative and quantitative data. Having one or the other is a very limiting factor and you may miss a key aspect of a certain robot that makes it either a great choice, or a horrible choice. That being said if your understaffed and you have to pick one over the other go with the quantitative. Quantitative data is unbiased and irrefutable. What did a team actually did, not what can the robot potentially do, not what the team said there robot can do, quantitative data tells you exactly how the robot performed. While qualitative data/ gut scouting is by no means bad there will always be some elements of human bias in this thinking and that can lead to problems when making a pit list and selecting people on the filed. |
Re: Understaffed scouting
Great points in here mainly figure out what/why you want to scout. Aside from collecting numbers what I find more important is being able to look at a field of robots and determine what type of robots we want on our alliance and then rank the robots we see on the field that would best fit those roles.
Prioritize robots that have strengths you don't have. Pick robots who will work well with you. If all three of you need to go to the landfill to get totes you'll all be in each other's way. The same goes for the HP station. The same can be said for autonomous: pick robots that have routines that work well with yours to achieve the highest score. Keep an eye on the field for consistent performers who can perform tasks reliably from match to match. Keep an eye on teams progressing as the weekend goes on. Some teams increase their performance near the end of their rounds and can be huge dark horses in alliance selections. Find out why teams are under/over performing by talking in the pits. Don't just look at the top robots on the field the more important partners are the ones who will be around for the last 8 picks and can often be the most important. Last year at early events it was hard to get a robot in the third round who could quickly and consistently gain possession for a three assist cycle or consistently score a ball in autonomous but a few were out there. Most of the time teams won't be in a position to pick the one of the top 2-4 robots at an event so don't dwell too much on them. Having the numbers on who scores more totes/rcs in auto and teleop is very important but don't just focus on getting numbers because you need to examine each robot through a variety of lenses that numbers might not show. In previous years teams who had a low average score could have been slaughtered with defense in their matches while your higher average teams never had defenders on them. Defense isn't applicable this year since alliances are separate but whenever a scout tells me "This team is really good they scored xxx on their own" I want to know if they were defended. Scouting with small groups isn't easy but its doable. You can throw 20 people at scouting but if they don't understand how to analyze a field of robots all the data in the world can't help you. |
Re: Understaffed scouting
1261 is at Palmetto (a 1st week regional) but is also sending 3 or 4 people to Perry (also a 1st week regional) who will be scouting in collaboration with team 2974 Walton Robotics. We will share data from Palmetto and they will share data from Perry, it is a win-win for both of us.
|
Re: Understaffed scouting
Quote:
Basically, if your scouts can't reliably count the number of totes a team can stack in a match, why would you trust that their "qualitative" assessments of teams mean anything at all? |
Re: Understaffed scouting
With a small team, it may be smart to avoid collecting a large number of repeats. Have a list of the teams you would like to collect during which matches to maximize the number of teams covered. Yes, you will miss some matches, but it's better to collect some data on every team than skip some teams that could have a vitally important mechanism.
Additionally, take pictures during your pit scouting operation. Pictures will help your scouters/drivers with recalling the specific bot, so having those available will help tremendously. Finally, as many others have mentioned, work with other teams in the area to scout effectively. Almost every team has some sort of scouting operation, and many rookie/small teams will be understaffed. Combining resources will be like coopertition, beneficial for all simultaneously. |
Re: Understaffed scouting
Quote:
That has put us in positions where we have no accurate data during an event which has been caused by technology failing or the input/collection failing. Either way I can still rely on my scouts to give me and our pick list makers a good evaluation of robots. Last year this played a critical role at two of our events and many before then where we had to say, "Data aside how do we feel about this robot from what we've seen". Like many have said in this thread you need to prioritize what you are scouting and why which is why we work hard to make sure our scouts know what to look for and why. If our data collection fails we have over half a dozen students who can offer their honest opinions regarding what they saw and work through their thoughts as a group to get a good feel for the field of robots. I think too many teams overlook the aspect of having a serious discussion regarding scouting and why teams need to do it which is why they get poor results. If you want to be successful at an event you need to treat scouting like you do the drive team in terms of importance and not a separate group in the stands. |
Re: Understaffed scouting
BrendanB's discussion of consistency in pick lists reminded me of this, so I'll just leave it here.
Quote:
Understaffed quantitative scouting has a tendency to be simultaneously boring and utterly exhausting, while not feeling particularly useful. It takes a lot of bandwidth to count game pieces and even more to track important quantitative movements. Qualitative scouting is like that, except it can be worse. Because (as you pointed out), if you're not well enough trained to count totes, it's unlikely that you're good enough with FRC to give meaningful qualitative input. That's the usual route. But--and I've been down this route as well--maybe you're understaffed but it's with trained, experienced scouts. It can be better to use those skills and keep them engaged with qualitative discussions than to bog down some of your best strategists in tote counting when you're not going to get good coverage anyway. It's all resource-dependent. By the same token, if you're trying to make a team culture that wants to scout, selling it as "sit here and count totes" can be rather trickier than "sit here and talk to me about matches. What do you see? Was that a smart move? What will they do in match 34? What should we ask them in their pit?" I struggled for years to build up a quantitative scouting system in a very anti-scouting environment. Qualitative is sometimes an easier first step. Not always, and it's not necessarily more effective in the short-term--in fact it probably isn't, but very little is effective in weak scouting systems anyway. But it's a way to fix some of the "just make up numbers" plague in match scouting. You're going to want both sides eventually. Scouting isn't just about how many totes someone scores. It's about predicting opponent's match strategies and individual play responses. Making a pick strategy isn't about ranking the highest scorers. It's about strategic decision trees and adaptability. In the end, blue banners aren't won by numbers on a page. They're won by allies that work well together, know their opponents and can manifest their work in their scores. |
Re: Understaffed scouting
Here's one scouting app that might be useful. Look for others as well.
http://www.chiefdelphi.com/media/papers/3098 Also, "gut" comes into play for pick lists no matter the scouting system. The arguments at our draft sessions are epic and famous on our team. We're all passionate and have great observations (just mine are better...;) ). Our final draft list may look little like our initial list, but we had a good starting point that we trust. |
Re: Understaffed scouting
Quote:
|
Re: Understaffed scouting
Quote:
This does 2 things 1> gets you the scouting data that you need, and 2> gets you to know (and hopefully become friends with) another team |
Re: Understaffed scouting
1 Attachment(s)
We too are an understaffed team in regards to just about everything. On our good days we have 10 kids. We are looking at having 2 kids scouting at a time and rotate them. I have attached the scouting sheet I came up with that allows you to scout each side of the field with a single person. I then have a Google form we will dump the data into that will take averages and show summaries for each team.
|
Re: Understaffed scouting
We plan to do "pit" scouting up front to decide which teams to watch closely. As we've built a landfill-miner, we will be looking most closely at the RC specialists, and somewhat less at the chute loaders. Rather than try to track each point scored (how do you score a stack with six totes from one team and an RC from another)?, we may depend on rankings and OPR from TBA for this data. Our match scouting will focus on reliability and clumsiness issues, and identify teams with effective vs ineffective littering.
|
Re: Understaffed scouting
Quote:
At Dallas, the mean score for an alliance was: Code:
46.1 total |
Re: Understaffed scouting
Quote:
At Auburn I'd suggest talking to NRG or 360, but I'm not so sure about getting outside help there (I dont remember any of those teams sharing scouting data in the past) Good Luck, and I'll see you next week :D |
Re: Understaffed scouting
Quote:
I'll spare you the entire description because (I already made a thread for that), but here is the down-low:
If you PM me, I'll help you out some more. Oh, and 360 is also using FRCScout.com and their lead scout is totally nice. He could also hook you up. |
Re: Understaffed scouting
We only watch one side of the field (red). With each team playing 12 times, we get 5-7 views of each team which is enough to make an informed opinion - plus we can get by with only 3 scouts.
Here is the sheet we've been using this year. We just log each match - draw the robot path on the right, graph the stacks and times on the left, and make any other notes on the top. It worked pretty well our first event, and we'll be using it with another team at INKOK. |
Re: Understaffed scouting
Quote:
|
Re: Understaffed scouting
The purpose of scouting is to be able to pick an alliance for Eliminations.
Since Coopertition does not exist in Eliminations, there is no reason to scout it. As I see it, the ideal alliance is: 2 stacking bots, and 1 utility bot. One Chute stacking bot, and one Chute or Landfill stacking bot. You know what your bot can do, so scout for the other two bots. You also want an alliance team that can do the following in autonomous: 1) Move to the Autonomous zone; 2) Bring along an RC (worth more than totes, and those you want to leave off the field to keep them out of the way). 3) If a team can stack the yellow totes in autonomous, that is a bonus. Don't try for a tote set. Not worth the hassle yellow totes in the way. For the utility bot 1) Can place the RC high (level 4 to 6) 2) Can move litter out of the way, but ideally to the landfill (each litter into the landfill is a 5 point swing). Litter: You want a person that is really good at throwing litter. If you find that person, make a note of which team they are on. Bonus points to that team. Given the slow pace of the game, it is possible for one person to scout all 6 teams at once. You may not have all the detail, but you can assign a robot to a role, and give it a score for how well it does that role. |
Re: Understaffed scouting
Quote:
1) Understands the game better 2) Scored fewer points in other areas Ignoring coopertition points will lead you to pass over these teams. |
Re: Understaffed scouting
Quote:
|
Re: Understaffed scouting
Quote:
|
Re: Understaffed scouting
Quote:
http://www.chiefdelphi.com/forums/sh...ing+the+app s |
Re: Understaffed scouting
Quote:
|
Re: Understaffed scouting
The way I see co-op, is most bots I see are NOT capable in the first place to do co-op at all ....so for a bot like us that is capable/versatile I feel strategically we need to get co-op out of the way early and then contribute with stacks RC's and noodles (take one for both alliances) ..that 40 is huge and omitting it is not going to raise your QA so in a way its sort of dammed if you do or don't...depending on how scouts view co-op. That part of co-op is quick and ROI is there.
I think its still good for a capable (similar to us) bot to go after co-op and if they are not alliance captain and alliances miss them then that's their (unpicking alliance captains) fault...and they missed a great partner that could be the contributing difference that wins in elims. They can shift gears in elims to HP stacker or superb RC+N topper. We hope to be alliance captain so we don't fall into that "faulty" co-op trap in scouting. I as a scout understand the value of co-op. Its basically the same action as other scoring maneuvers. Unless you are strictly an HP tote stacker. I know what our bot can do and it would be a shame to scout a bot out over co-op in the versatility aspect...unless that's all they do which is unlikely. |
Re: Understaffed scouting
Quote:
Quote:
Quote:
|
Re: Understaffed scouting
I see a strong alliance as one that has two stackers, and one utility bot that can place RC's.
Just because a team can place 2 or 3 totes on the step does not mean it can: 1) Stack quickly; or 2) Place an RC on top of 4 to 6 totes. If a bot spends the entire match placing 2 or 3 totes on the step for coopertition points, I would not consider that robot good for either stacking or utility. Now, if that same bot placed the yellow totes in 30 seconds, and started making other stacks or capping, then it would get noticed for that. |
Re: Understaffed scouting
The purpose of scouting, as with all competitive efforts, is to win the competition. Good scouting helps win qual matches, not just elim ones (via alliance selection and beyond).
I've seen a lot of teams in Week 1 and 2 blow repeatedly co-op (and often therefore their ranking) just by setting the wrong coopertition strategy. Accurate analysis of previous efforts would've been more than enough to correct the poor plan. You have to accept trade-offs when you're understaffed, but carefully consider the consequences of each one that you make. |
Re: Understaffed scouting
Quote:
|
| All times are GMT -5. The time now is 17:25. |
Powered by vBulletin® Version 3.6.4
Copyright ©2000 - 2017, Jelsoft Enterprises Ltd.
Copyright © Chief Delphi