Thread created automatically to discuss a document in CD-Media.
Team 33 Scouting Sheet by: IKE
This is the scouting sheet used by Team 33. For the district events, we would scout every team, every match for 6 mtaches. At the Championship we chose to skip the first match and then scout the next 5.
This is the scouting sheet used by Team 33. For the district events, we would scout every team, every match for 6 mtaches. Each team was assigned a sheet and we had 6 students scouting the matches and 1 master scout (actually 2 that traded in and out) that arranged the sheets for the next match, and sorted the returned sheets. At the Championship we chose to skip the first match and then scout the next 5. We would then compile the data into a score on “positive actions”. Passes (over bump) and Goals counted equal, and hanging was worth 2 pts. This overall score often ranked very well with our alliances selection list. the important thing with this metric, is that it still takes into account scoring/passing for the other team. On our sheets this is the smae as scoring for your side. We found this to be more accurate than OPR for many of the great teams offensive potential.
Having the seperate sheets was great for arguing rank in alliance selection. You could pull up someone with a high score and see a cross section of 5 matches and get a feel for striker/winger and how consistent they were. The sheet also contains additional categories of information including autonomous and subjective evaluations of speed and driver skill. The Autonomous section is great for arguing positions within an alliance. Many teams claimed making every ball in auto-mode. Having a sheet that clearly shows them only being able to clear 1 of 3 was very valuable. The subjective sections are somewhat un-reliable. Since sheets are handed out randomly to scouts, one scouts med speed may be another scouts slow. If all 5-6 scouts agree it was slow, then it was slow.
This is the scouting sheet used by Team 33. For the district events, we would scout every team, every match for 6 mtaches. Each team was assigned a sheet and we had 6 students scouting the matches and 1 master scout (actually 2 that traded in and out) that arranged the sheets for the next match, and sorted the returned sheets. At the Championship we chose to skip the first match and then scout the next 5. We would then compile the data into a score on “positive actions”. Passes (over bump) and Goals counted equal, and hanging was worth 2 pts. This overall score often ranked very well with our alliances selection list. the important thing with this metric, is that it still takes into account scoring/passing for the other team. On our sheets this is the smae as scoring for your side. We found this to be more accurate than OPR for many of the great teams offensive potential.
Having the seperate sheets was great for arguing rank in alliance selection. You could pull up someone with a high score and see a cross section of 5 matches and get a feel for striker/winger and how consistent they were. The sheet also contains additional categories of information including autonomous and subjective evaluations of speed and driver skill. The Autonomous section is great for arguing positions within an alliance. Many teams claimed making every ball in auto-mode. Having a sheet that clearly shows them only being able to clear 1 of 3 was very valuable. The subjective sections are somewhat un-reliable. Since sheets are handed out randomly to scouts, one scouts med speed may be another scouts slow. If all 5-6 scouts agree it was slow, then it was slow. We always use a comments section to help cover certain unpredictables like DOAs, dying mid match, flipped by opponent…
I will attach some of the “completed” sheets once I have redacted team numbers and some tough comments. We ask our scouts to be brutaly honest in their assessments, and sometimes that is tough for others to see.
Something to consider though - scouting every match. In 2007 we had a selected a team that we worked very good with in Match #2 of the regional. Their first couple of matches afterward were also very good and we stopped watching them after their 3rd match because we knew enough about them. The problem was that after their 5th match they bent their arm every so slightly that it wouldn’t grip the tube anymore. They were useless to us. We asked them if everything was functioning properly and if there was anything broken; they said they were good to go. We selected them and soon found out they weren’t in working condition.
Point being, we now scout every match and inside of a database keep a progression chart based on their average alliance score and opposition average score. Just a word of caution.
Interesting. Do you have the same 6 students scout all these matches? It seems if you didn’t, your sheet might be too qualitative to get completely accurate data. Of course, you have students who feel that Team X’s accuracy is about a 4, but another who thinks it a 2. As compared to a score/miss (which there really aren’t “misses” this year, mainly just moving balls between zone) which would give you an accurate data every time, as compared to a student’s discretion. Just a suggestion, although they are nice sheets. You collect a lot of data every match.
Do you put these things into a computer/program? It’s an easy way to organize and pull data quickly on select teams.
The sheets are interesting. The data seems slightly too qualitative for some things. Seemingly, a more logical and efficient scouting system would be entirely based off of a quantitative data set. Unless you have a single person assessing every single team, those rankings can be pretty much thrown out, because what one person gives an eight on a scale of one to ten, another might give a 6. It is all based on opinion and predisposition; indeed, the biggest problem with this system is bias: where one person might tend to be more lenient with their ranks, others might be much more hard to impress; therefore, your data can vary extremely. This is dangerous to do, and can also cause disputes of opinion. Pure numbers are the only way to truly have irrefutable evidence.
Another comment, I would like to agree that scouting EVERY match by a team is a much more reliable system. This system would help to make averages more accurate, and would give you less flawed data. Take for instance the possibility that a team has comms issues at the beginning of the competition (I know this happened to us, 2337, at Kettering), and then after their fourth match the team begins playing EXTRAORDINARILY well. You’ve then forfeited accurate data for their last six matches, which may show that they are a team whose scoring averages (or other relevant data) would have otherwise put them at the top of your list.
You are correct on the qualitative issues of multiple scouts. We have about 10 students that rotate through the scouting position, and 2 head scouts that do the sorting. Generally the only time we see big discrepencies on qualitative metrics is when a team is extremely inconsistent. This usually is mirroed in their quantitative metrics as well. With consistent teams, we may see an oscillation between 3&4 or 4&5. These qualitative metrics are more for general observation that making an overall decision about a team.
As far as scouting every match, we do. We just don’t use the sheets for every match. By end of the day Friday in Michigan, we have seen 8 matches and have data on at least 6 of them. This is enough to see trends of performance and do a general ranking. We rank the teams and make comments on them about the general abilities and style of play. There are 2 master scouts complimenting this system. Myself and Chet (the nicest Grumpy Old man you will ever meet). We have a list of teams to follow and what we are looking for from them. I will often go talk with the teams Saturday morning that I feel will be in key picking positions or that we need unique strategy from. For instance, I asked 1622 to show me their home row Automode as we were interested in them as a Striker. We aske 148 what had been going on during the tournament (they had a bad Friday) and asked them to showcase certain abilities on Saturday (and boy did they deliver). There were other teams we were considering that frankly fell flat Saturday morning. You can PM me if you are curious if you are one of those teams.
This system has worked very well this year. At the districts were were spot on for who would be playing in elims. The order was frequently different, but the teams playing were not. At MSC, we went 23/24 out of 65 teams playing in Elims. The team not on our list fell over twice in the two matches that their alliance was defeated in. In Archimedes, we were 20/24. There were a couple of surprises in the top 8, and a couple of picks I just didn’t (and still don’t) understand.
All this being said, we had a major issue at the Championship. The picklist that we spent 2-3 hours making Friday night was left at the hotel. I thought the head coach had it, he thought I had it. Luckily with the sheets, we were able to reconstruct the list in about 20 minute right before alliance selection. We will add “Make Multiple Copies for Head scouts, Coach, and Team captain” next year. No matter how robust the process, execution is still key!
As part of the scouting team I can assure you that every robot is well looked at. Although by the end of Friday or early Saturday we have 6 matches on every robot, we are watching every match and will extra scout teams who we’re not completely certain about. If some team is not well represented on our scouting sheet, but they play a good match later, we take that into consideration. In addition we also pit scout every team. By the draft we have a complete list of teams we want to pick in order. Then the Captain can just cross them off as their picked and pick the next one that isn’t when our turn comes around.
We have our whole team (about 30 people) cycle though 6 scouts every 16 matches (each student ends up with a few scouting times a piece), and then there is myself, who cooridnates scouting (gets sheets out, collects sheets, inputs data, watches a few matches, ect.) and one other with me who also helps enter data and other misc. tasks I need help with. In addition, we have 6 people who watch matches (5 mentors, and myself, not always taking notes full time) taking notes on pre-selected teams (selected based off of high scouting data, or teams that would compliment us). We then bring back our laptop with the data, the notes people took, and the guy who records all the matches (so we can reference them during the meeting, but record them to post on TBA/YouTube), and we talk about possible picks. We say a team number, we compare notes, and we rank them. The next day, we keep watching and before alliance selection after our last match have a quick meeting to finalize our list.
That’s awesome! I’ve yet to get as close as 23/24, but we have gotten around ~19ish.
Oh good, so were not the only one with long scouting meetings! Ours usually take about 1-2 hours, depending on how in-depth we go with each team and the discrepancies with teams we scouted, and where to rank them.
Also, it’s important to note that what is looked at changes from game to game. For instance, in Lunacy, human players had a huge say in the outcome of the match, so they all had to be ranked too. I must say, it was extremely cool to see all the scouting data about myself. And read the comments about me.
Ours usually take about 1-2 hours, depending on how in-depth we go with each team and the discrepancies with teams we scouted, and where to rank them.