How 492 won the PAC NW regionals
 

Something is very different in this years game than the my knowledge has never happened before (or at least no recently).

3v3.

Though this seems like a pretty obvious thing, very few teams took it into account any way other than robot design, which even then only a few did.

Don't get me wrong, team 492 's bot Tyr was a very well designed bot, and I believe with that alone we could have placed very highly, it is not what made us win the Pacific NW regional almost unchallenged.

No, I would attribute our sweep of the PAC-NW to two other, though often overlooked, reasons, namely scouting and strategy.

Scouting:
To get truly accurate data people had to change their scouting this year. With a 3v3 game match ranking was just a horrible way to judge teams. So we made our own.

492's scouting this year I have to say was the best that I myself personally have ever seen. As soon as our robot was shipped we started working on our scouting application. We made a php based scouting application that allowed our scouts to input data from every match. Here is a screen shot:
http://trc.freedomdown.net/scout1.gif
We then coded a php info script too allow us to make sense of all our raw data. Screen shot:
http://trc.freedomdown.net/scout2.gif
In the app you can sort be what ever you want, call up data on what ever team you want etc.

At regionals we set up a wifi network with a laptop server running in the stands.

We assigned 4 team members full time scout duty. These members dutifly recorded data for every match in the entire regional.

This allowed us to find the truly two best teams in the regional to partner up with, team 1595 and team 604. Due to 604s ranking they were not picked as first pick by any other team, even though they were one of the best. We ended up getting our first two picks, and getting an owning alliance due too our scouting.

Strategy:
Sadly not until after nationals can I really go into what strategies we used that made our alliance so effective, but all I really can say is that with this 3v3 game, it changes allot. You have to know how to exploit being able to use all 3 robots at once to do different things, not have everyone trying to do the same.
One of the reasons that 604 had not done to hot before finals even though they had a really nice bot and very good driving is that they had no method to their madness, they could cap very well but just didn't know where. When we hooked up with them for the finals we made this all very clear.

Shout out too 1595 and 604, you guys were some rocking alliance partners, able to get a 40-40 tie going 2v3 without is in finals!

Re: How 492 won the PAC NW regionals
 

That is a very impressive piece you got there. Great work on your scouting. Scouting can do a lot in a robotics competition and help make the best decision about strategy.

Re: How 492 won the PAC NW regionals
 

It was a great scouting system I think also, even though I am not sure where some of the information comes from because more then 3 or four times we had more then 3 tetras capped in a match. We consintently had tetras on top of the center row and the we would be able to cap some of the back row also. I know our max was not 3. Everything else seems very good. I was trying to figure out the quality and schedule rankings but I'm really confused where those numbers came from. We could see why you picked 1595 (consistant auto mode, 4 points off the bat, and then 4 or 5 caps a match it was awesome.) I was told at regionals you guys weren't going to make it to nationals, is this true. You will be missed if it is. What are the stats on 847 (#2 alliance captain at regionals)? They seem to missing from the top of the list and I wondered what thier stats were like)

Re: How 492 won the PAC NW regionals
 

Hehe, when we were going over this data back at the hotel we were saying the exact same thing. How TF could our max tetras be sooo low? But then we went back and did the math, we even reviewed pictures we had taken with a really high quality digital camera to count them up. Something you should know though, those are just In-Game tetras, so any capped/dropped in autonomous are not in there. But after much review we found that actually that data was correct. It appeared that everyone thought that they were capping more than anyone really was. Right after the match ended all 4 of our scouts double checked to make sure that the number of tetras on the field accumulated to the number we had gotten for both alliances.

Schedule is how hard their schedule was, based on how good the teams were that they were going up against.
We are. We weren't until we won regionals, and decided when we got back too the pits. Exactly what happened last year too...
Here is the same image with a little more high, so you can see 847, the bottom team.
http://trc.freedomdown.net/scout3.gif

Re: How 492 won the PAC NW regionals
 

Well so far, we have register with FIRST and it looks like we are going to nationals, but that really depends on if we can fundraise several thousand dollars by the time the money is due.

Hopefully we'll make it.

Re: How 492 won the PAC NW regionals
 

Good luck, we will see you there

Re: How 492 won the PAC NW regionals
 

Correct. During the regional a man from NASA came up to us. He talked with some of our mentors, and offered us a grant up to $5000 pending a budget. Well we only found out weeks later he meant a matching grant, so we are going to go into the fund raising furry.

Re: How 492 won the PAC NW regionals
 

That kind of upset occured at BMR as well. Team 93 had a 1-9-0 record, and nobody picked them except for Team 135. This alliance went on to become finalists.

Congratulations on your scouting. 461 used a similar system. We however assigned 6 students to be field scouts, one for each station, and 1 student to be a runner between the data input area and the stands. This way, things were a bit less hectic in the stands for entry and people could focus on one bot at a time.

Re: How 492 won the PAC NW regionals
 

No this intruiges me. Did you use a web based (php) app such as ours, or did you use exel (like we did last year)?

Re: How 492 won the PAC NW regionals
 

We (461) did extensive scouting at the Boilermaker Regional.

We used the following for our scouting:
* Microsoft SQL Server for data storage
* Microsoft Access for data entry
* PHP for making our data available cross-platform as a website

I am in the process of adding a web page to our site that explains this in more detail. I will post the page as soon as I finish it (most likely some time today).

If you are going to the Midwest regional or you are in our division for Nationals we are looking for teams that will join with us so we don't each have to do our own scouting. If not, you can contact me and you may be able to use our software for scouting. I will be posting a new thread explaining this further here on Chief as soon as I get that web page written up.

Re: How 492 won the PAC NW regionals
 

What made you choose to you access rather than a php app to do data entry?

Re: How 492 won the PAC NW regionals
 

Impressive scouting method. Nice work, and you certainly picked good partners for the finals.

I wanted to take this opportunity to point out to the utes here that purely quantitative analyses have to be subjected to some critical thinking before they are accepted. Since we were at the PNW regional, I'm going to go ahead and use that as an example.

1. Your quant analysis draws some misleading conclusions. Using my own team as an example, your quant analysis shows that we scored between 0 and 4 times per round, and you averaged that out to 1.6 per round. This analysis using the mean (average) number neglects that fact that our scoring did not have a normal distribution. We had an arm fail in three rounds (twice due to a PWM cable being knocked loose before the match, and once when the arm operator jammed the arm by moving it into the stop when our software limit detector had been accidentally overridden by some code changes). This means that we scored 0 tetras in three early rounds, but scored between 2 and 4 times in the other rounds. A more useful measure might have been the median (center value when ranked) or even the mode (the most common result) rather than the mean for this particular measure. Using 1294 as an example, our average score was 1.6 capped tetras per round. Our median was probably 3, and our mode was probably 3 or 4. My records aren't complete, but this is pretty close. This means that when our robot hadn't been sabotaged by our own team, we reliably capped 3-4 tetras per round. Your quantitative analysis missed this. (Now you could make the argument that being unreliable should count against a team, and I would agree with you. That's not my point. My point is that, by its nature, a simple formula cannot take everything into account.)

Another way your Scout might have looked at an event like this would have been to qualitatively note that a robot which performed well at the Bellevue trials and on practice day failed to perform early Friday, and then worked fine after that. This is not an argument that we should have been chosen for the finals, but rather that I wanted to show you that a simple quantitative analyses won't always tell you what you think they will tell you.

2. You have no sense of time series in your data. Bots (like 997) that started off really strong ended up fairing comparatively worse as others learned to defend against them. Likewise, other teams became stronger as their driving teams got better. By treating all data the same, you probably over-emphasize early results. Try a weighting factor over time next year and see if it changes your analysis.

3. Have you done sensitivity analysis on your metrics? This means that you should play with your data to see if fairly minor input changes result in large outcome changes. I'm not asking you to tackle serious multiple-goal multi variable equations here, but you should know in advance if your model is robust enough to only reflect small ranking changes with small changes in the input data. Some models in the world fluctuate wildly with small input changes.

4. Some of your data are wrong, probably because of under-sampling. What process do you use to collect and quality-control it? You probably want to make sure to have different scouts evaluate each robot. As an example, in most of our matches we started off with a held tetra, yet you say "no" to this in your spreadsheet.

5. Unlike a baseball statistical analysis, your universe of measurements is too small to be statistically significant. This means that you should always apply human analysis before accepting the results. (You may, in fact, do this. I just wanted to encourage all teams not to blindly accept nice-looking quantitative results that may actually mean nothing.)

Your methods are proven by your results. You went through the finals like Patton through France. As I said up there, I am just encouraging all scouting teams to not just trust their numbers without fully understanding what those numbers mean and where they came from.

I think you did a nice job.

Re: How 492 won the PAC NW regionals
 

While this thread and its data are rather interesting, I don't believe that the actual contents of the message (that is to say, that strategy and scouting are fundamental to winning a regional) are any different in past years. It only seems like a focus this year because of the fact that there is only one field object to manipulate, as opposed to a variety last year. Every robot has the same basic function. Scouting is always going to be important and I am certain that every successful team has its own individual methodology when it comes to this fact, that isn't to say that 492's system isn't impressive; rather it is another piece of the puzzle. At the same time, strategy should also always be a given when it comes to success in the qualifications and the eliminations. It isn't un-GP not to share strategies especially because of their heightened importance this year with only two ways to score. It takes effort to think about and determine what the best course of action is. I would argue that flexibility is the key point this year. Stacks and rows can change hands quickly, and it takes a fast, analytical person (luckily, our field coach happens to be one :D) to make a decision on the fly. Strategies are important as well, but they are not the be all and end all of this year's game.

This also brings up a good point. Do not discount the scouting and strategy members on your team! Some of them may not have done hands-on work with the robot, but if they are able to tell you go to "X" goal and cap it and win the match, then they are as valuable as someone who spent the time building the arm that did the capping. Far too often there is a perceived notion that strategy and scouting represent the "soft" parts of the team, but let us all remember that with an efficient and effective system, a seemingly small time alliance can work its way up to become giant slayers.

Re: How 492 won the PAC NW regionals
 

Great job on winning, 492! Such a scouting database is quite an impressive feat. Having scouts focused enough to keep such a complete record is no doubt a challenge, and very nice work on the backend too. Our plan was to do something similar with manual input to a spreadsheet, but our scouts in the stands didn't come through with the numbers. (Although to be fair they weren't entirely informed that we wanted to compare them.) To see how big a difference such good scouting can make is really inspirational, though.

Re: How 492 won the PAC NW regionals
 

That is definitely a very impressive strategic aid, and if it helped you like that I'm very interested in better definitions of the different columns, after nationals of course.

Re: How 492 won the PAC NW regionals
 

My point was that this year a certain type of scouting was important. The type that required individual scouting rather than going by ranking and scoring.

In our newest version of the scouting app we take into account mean, median , and mode. Also, the reason we have all those columns is precisely that, no one formula can take into account all the aspects of the game.

Weighing all data equally requires a consistent bot, I will grant you that. But do we really want a bot that only worked well for the last 4 matches?

Collect: we had 4 scouts covering every match. 1 for each row, as well as 1 for entering data.
For quality control: well even though each person was assigned to a row, they were always keeping an eye on the others, just to make sure know one had fallen asleep at the wheel.
If we say "no" for your team it was only because that was what were told by your team. That n/y was recorded PRE games on thursday, so I guess one of your team mates was confused.

Well we went on the data we had to go by. Of course if we had more data it would be more accurate, but that is just wishful thinking.
Thanks.

Re: How 492 won the PAC NW regionals
 

Scouting is very important part of the game every year. Over the past 8 years I believe our team has come up with a very effective scouting method. On Thursdays/Fridays of the competition teams of students are have a set group of teams they are to be scouting. They are required to get pictures and also watch all of their practice/qualifying rounds. While they are watching they fill out papers with different questions based on their performance as well as overhead sheets showing the path of their auton. This way we can overlap the overheads to see if they made any changes to their auton during the day. Then they go down to the pits and ask the same questions that they filled out while watching the teams to the people on the team to see if the information provided matches what they saw on the field and also to get any information they missed. All of this information is then entered into a database that we have.

Before each of our rounds we have our head scouter bring us our "cheat sheets" on each of our partners as well as our opponents as well as a proposed strategy based on the teams in the match. The drive team then takes the sheets and goes to see our opponents robots to see some of the technical aspects of the robots that the scouts up in the stands my not see(i.e. 6 wheel drive-can we push them/push us etc...). Then based on all the information provided by our scouts and the drive team, the drive team then goes to talk with our partners with a proposed strategy, if everyone is in agreement then thats what we go with, if not we make changes as necessary to come up with the best plan possible.

Re: How 492 won the PAC NW regionals
 

It is really easy to build a powerful GUI for data entry with MS Access. Basically, we saved ourselves the time and effort to create a PHP data entry interface. MS Access also provides a much nicer and faster GUI than an HTML page. This helps in navigating quickly through records.

Re: How 492 won the PAC NW regionals
 

Heh, I agree with you there, but what I was saying was that this type of "individual" scouting has always been used by any successful team to get where they need to be. Ranking and scoring is never enough. Reliability, consistency, even the geniality of the coaches and the field crew all play an important role in getting together a successful alliance.

Re: How 492 won the PAC NW regionals
 

Yep, just more important this year. This year your match results are influenced by 5 other teams per match rather than 3.