Computer-Assisted Scouting

[FourPenguins]

MORT made some tentative steps into the brave new world of non-paper scouting this year, with a very buggy, student-developed scout program that never made it to competition.
Despite the fact that this software wasn't ready in time for competition, it raised some interesting points about the benefits and disadvantages of computer-assisted scouting and paper-based scouting.
Who else uses computers to help scout, what software/technologies do you use, and how do you think the system has helped or hurt your scouting abilities?

[anna~marie]

In the past HOT has used a collection of PDA's... it worked pretty well but it was too hard to implement this last year
It was very nice to keep things organized and statistics were so easy to figure

[Kristian Calhoun]

You may also want to take a look at this thread (http://www.chiefdelphi.com/forums/showthread.php?t=46280&page=1&pp=15) which discusses paper vs. computer scouting and the different methods which teams use. This past year, our computer scouting program was written in Java. Scouts recorded the statistics of each robot per match [each student was assigned a specific robot starting position (i.e center red, left blue, etc.) and at the beginning of each match, they'd record the team number, and fill out the sheet for that particular team's robot throught the course of the match. The papers would be collected and immediately entered into our scouting program. Once inputted, the program would analyze the data and could sort it into whatever form we needed (specific individual team stats, power rankings and lists of rankings based on different things such as number of 3 pt balls scored in auto, number of total balls scored (3pt/1pt), wins/loses, percentages, etc).

Besides match scouting, we also had scouts go down to the pits with digital cameras and take pictures of each teams robot. We then imported all the pics into Google's Picasa and labeled them.

This method of scouting proved to be extremely reliable, helpful, and accurate. At any time during the competition, the drive team could call a member of the scouting crew, and ask for any kind of information. We could then either tell them the info over the phone, or print it out and have someone run it down to the pits.

This is what our head scout, Neil Parikh, had to say about our scouting program in the aforementioned thread which I linked to.
Team 25 has finally moved into the future. For as long as I've been on the team, since the '04 season, we've used regular paper scoutings sheets + a filling bin...yay for that.

But after getting frustrated with having over 1200 sheets of paper to store, etc, by the end of 2 regionals, we've finally moved over to a paper/computer system.

Yes, our scouts do record information on scouting sheets (one for the match, one specifically for autonomous)...then that data gets fed into our amazing Scouting Program for analysis, output, and just about everything else under the blue moon!

This year I happened to notice that about 5 of us Robotics kids were in the AP Computer Science Class, and I figured, what a neat way to get hands on experience coding Java & we'd get a really neat scouting program. Regardless, we worked tirelessly with the help of Mrs. Morris (our computer science teacher who spent loads of time designing + perfecting it), it was done in time for the NJ Regional.

In addition, we have 2 video cameras taping matches and a photo team that gets pictures of every robot. These are fed into Google's Picasa for labeling, viewing, and searching.

We pride ourselves on the fact that about 95% of the data we collect is pure statistics, leaving out a significant portion of error and uncertainty that occurs with opinionated scouting. Luckily, I've got about 8-10 people that work hard and long all through the competition making sure we have accurate data, footage and pictures.

When the system was finally put into use at NJ & Las Vegas, it proved to be incredibly successful in helping us find the best robots, and also for helping to develop strategy for rounds. At both regionals, we made our scouting data available to any team who wished to take a look through it.

So yeah, we use computer + paper scouting. Paper means we always have a backup. Computers mean we have amazing speed & efficiency when looking up information. I don't think we'll ever look back

[SamC]

This year our team used paper and computers for scouting. We would record data on paper then pass it on to a crew of people who entered the data into an Excel spread sheet. That was convenient as you could sort it by Match, Team, and different statistics (shooting accuracy, defense, offense, # of balls picked up, etc...).

Earlier in the season a few mentors and students tried to make it an all computer system but it wasn't finished in time so we integrated the two together. Maybe next year it will all be ready!

[FourPenguins]

You may also want to take a look at this thread (http://www.chiefdelphi.com/forums/showthread.php?t=46280&page=1&pp=15) which discusses paper vs. computer scouting and the different methods which teams use.
Thanks for the tip. I did read that thread before posting and it was very informative, but for reasons of topicality decided that further inquiry should get a separate thread devoted solely to computer-assisted scouting.

[Starke]

Being a first year mentor (student on the team in previous years), I was able to teach students how to scout effectively. The students and I made a very large excel spreadsheet. It is similar to Greg Needel's scouting spreadsheet, with some modifications to better fit what we needed. I brought my laptop to The Championship, where some students and I went around and asked questions and punched the answers in the laptop. The students and I also found pictures of most of the robots before The Championship started, and we stored them in the laptop as well. It was great to have everything in one small place. And it came in handy for the Einstein Field :yikes: (ask 1126) :yikes: .

[AdamHeard]

You may also want to take a look at this thread (http://www.chiefdelphi.com/forums/showthread.php?t=46280&page=1&pp=15) which discusses paper vs. computer scouting and the different methods which teams use. This past year, our computer scouting program was written in Java. Scouts recorded the statistics of each robot per match [each student was assigned a specific robot starting position (i.e center red, left blue, etc.) and at the beginning of each match, they'd record the team number, and fill out the sheet for that particular team's robot throught the course of the match. The papers would be collected and immediately entered into our scouting program. Once inputted, the program would analyze the data and could sort it into whatever form we needed (specific individual team stats, power rankings and lists of rankings based on different things such as number of 3 pt balls scored in auto, number of total balls scored (3pt/1pt), wins/loses, percentages, etc).



Do you have any plans to release this program to all teams?

[dhitchco]

Most teams now do some sort of "computer-assisted" scounting which can be anything from
a) manually-tabulating scores from paper-based sheets into a Excel spreadsheet to
b) directly-recording scouting info into a PDA to
c) our team's use of optical recognition software to automatically scan and summarize all the scouting sheets into a database.

What kills me is all the teams running around taking photos in the pits of all the same robots. Isn't FIRST supposed to have a central database of all 1,000-plus robots prior to the first regional event? Oh well, maybe somebody else can really get this going better. Maybe even a photo kiosk at the championship?

The real trick to scouting (pit or match) is to make it SIMPLE and truly meaningful when picking alliances. Too much data input just confuses the process. Keep it simple.

[kramarczyk]

The gearheads (1189) have been using a electronic system based upon PHP/MySQL. Additionally, we have been providing access to the system to everyone in the arena via a wi-fi network we set-up. An example of the system provided at the 2006 Waterloo regional is available at:
http://blarg.moonoo.com/crossing/

A system such as this provides an opportunity for collaborative scouting. I realize that certian aspects of scouting are propriatary, but for definable metrics this would reduce the burden on each time. Thoughts?

[nparikh]

Do you have any plans to release this program to all teams?

We'd love to--but we've faced a number of set backs in that a) most people wouldn't understand how to use it b) it's written in console Java and so most people would have a hard time getting it to run unless they have JCreator on their computers already.

Perhaps one of our Java programmers can do so ;) cough Alexa cough.

[Tim Delles]

Team 229 has always used paper when doing scouting (and mostly the mind as well)

This year it is my hope that we can switch over to the wonderful world of computer-assisted scouting (in one format or the other).

I have looked into this in the past and as has been said there are both benifits and consequences to using computers instead of the good old paper and pencil routine.

Good luck in whatever you all choose to use.

[[527]phil]

I'm experimenting with a scouting program that I'm developing myself. I'm still a noob in VB but I'm learning at a fast pace. Basically what I'm doing is making a GUI for both a laptop, and a PDA running windows pocket 2002. When you input data into the program, it places it in an excel spreadsheet that can be transmitted Via a wireless connection to the PDA, or vice versa. that way the scouter's in the stands can just send the new data right to the pits.

EDIT: A few people told me that Wireless networks are banned at the competitions, so I'll have to figure out something else.

[Dylan Gramlich]

Being a first year mentor (student on the team in previous years), I was able to teach students how to scout effectively. The students and I made a very large excel spreadsheet. It is similar to Greg Needel's scouting spreadsheet, with some modifications to better fit what we needed. I brought my laptop to The Championship, where some students and I went around and asked questions and punched the answers in the laptop. The students and I also found pictures of most of the robots before The Championship started, and we stored them in the laptop as well. It was great to have everything in one small place. And it came in handy for the Einstein Field :yikes: (ask 1126) :yikes: .
I must say i was quite shocked when you whipped out the laptop like a ninja and gave us the lowdown on the strategy to beat the newton alliance(which worked for one match ;) but then :yikes: they changed there strategy and we broke...). But the simple program you had on there looked quite helpful and easy to use.

[ewankoff]

our school district issued every student their own laptop. in the past years we have tried to develop programs that each scout could use their own laptop to update a scouting database. unfortunatly we had problems integrating these programs with our team ad the network and they did not work. last year we combined paper scouting with a computer database. scouts would fill out of sheet of 4 matches where they scouted the same spot 4 times. then the sheets were run up to a person inputing it into the excel spread sheet. this method helped us get into the finals in palmetto last year.

[Fred Sayre]

I started to talk to the guys responsible for this:
http://stampscouting.org/

It is a collaboration of PNW team members to great a web browser based database to be used with networked computers. The concept is great but there were too many late minute issues to be worked out for us to use the system.

As a alternative, we created an access database, you can check it out:
http://students.washington.edu/fsayre/post/ATLScouting.mdb

It has a robot-match input form that represented all of the data for a match per robot, and mimicked paper forms that our scouts filled out during the matches. That form fed a stats table that stored the information. We then used two queries to represent the data. One calculated averages of each field for each robot - Avg shooting score, autonomous score, penalties, times that the robot made the ramp etc. Also we used a query that basically represented most of the data that we collected so that we could look at a team's performance on an individual match basis. If you know your way around access it is pretty convenient, you can right click on the team field and filter by a certain team, sort by match number, and get a good impression of how the teams were doing. We were able to accurately predict which opposing alliance teams posed the biggest threat :)

[Graham Donaldson]

our school district issued every student their own laptop. in the past years we have tried to develop programs that each scout could use their own laptop to update a scouting database. unfortunatly we had problems integrating these programs with our team ad the network and they did not work. last year we combined paper scouting with a computer database. scouts would fill out of sheet of 4 matches where they scouted the same spot 4 times. then the sheets were run up to a person inputing it into the excel spread sheet. this method helped us get into the finals in palmetto last year.

Yeah... and it was me and joe putting all that in! That was terrible! Don't recommend doing that again.

As a alternative, we created an access database, you can check it out:

That's what I'm planning to do this year with scouting. I'm going to be running a Microsoft Access class during our training sessions (I learned it over the summer), and we hope to build a database with forms and everything and then hook it up to our wireless server (no internet, just server-based forums and such we set up for each regional) so that with our laptops (which have wireless built in), have round scouters fill out the forms, send them to the database, and have a live update. Someone will be up top overseeing everything, and sending reports down to the pits either by a forum post or by a printer sheet, or a telephone call. Hopefully, I'll be able to post the database on Delphi once it's finished. But we'll have to see what happens.

The Canuck

[Dylan Gramlich]

On SparX we write down scouting information on our "scouting cards", these also have pictures of the robot on the back. We then input the information into an excell spreadsheet which organizes the data very nicely so that we can then narrow down our choices if we are picking.

[Graham Donaldson]

On SparX we write down scouting information on our "scouting cards", these also have pictures of the robot on the back. We then input the information into an excell spreadsheet which organizes the data very nicely so that we can then narrow down our choices if we are picking.

We used Greg Needel's Excel program last year (worked very nicely, got us into finals at Palmetto *cough* 50% Scouting *cough*), but the problem with Excel is input is slow (you have to manually go from box to box) and navigating through all the spreadsheets for teams is time-consuming. With Access, we can create a drop-down menu for all the teams, and customize the tab-advancing layout of the input forms (ex. what box you'll type in next when you press the tab key). We can also print out copies of those forms if we have to do manual writing, but I've been told by the team website/internet/server/email/technology in general guy that we can set it up, as I said above, with a live update so once they send it in it will go to everyone connected with permission to the server and has the file, which is easier than last year. We had a huge backlog by the end of the qualifiers day. We ended up only doing the rest of the inputting for the teams we would be facing the next day, and then scrambling to put the rest of the info in for the teams we would face in the finals with 95 and 16 as alliance partners (thanks again guys!). I'm hoping the Access program will work much better. If we see you at a regional and you have Access, we'll be happy to give you the file (not the info! Well, maybe- TBD), and again, I'll try to post it on Delphi when it's finished.


The Canuck

[FourPenguins]

phil']
EDIT: A few people told me that Wireless networks are banned at the competitions, so I'll have to figure out something else.
A quick search of last year's "At the Event" section has yielded no such rule. Does anyone know the rule number or what section I can find it in? I seem to remember people picking up on WiFi networks being run by teams last year at competitions (Nintendo DSs are excellent network sniffers.)

[Starke]

We used Greg Needel's Excel program last year (worked very nicely, got us into finals at Palmetto *cough* 50% Scouting *cough*), but the problem with Excel is input is slow (you have to manually go from box to box) and navigating through all the spreadsheets for teams is time-consuming.The Canuck

I found this problem as well. The students an I came up with hyperlinking to each team page from a main page in the same document. On each team page, we also hyperlinked back to the main page. This took a lot of time to do, but it saved a lot of time in the end.

[Graham Donaldson]

A quick search of last year's "At the Event" section has yielded no such rule. Does anyone know the rule number or what section I can find it in? I seem to remember people picking up on WiFi networks being run by teams last year at competitions (Nintendo DSs are excellent network sniffers.)

We had no problems setting up our network at Palmetto; even got a couple of teams using it. Maybe the ban on WiFi relates to networks with internet access, and not simply a server, like ours?????

The Canuck

[Sean Schuff]

I started to talk to the guys responsible for this:
http://stampscouting.org/

It is a collaboration of PNW team members to great a web browser based database to be used with networked computers. The concept is great but there were too many late minute issues to be worked out for us to use the system.

Team 93 had mentors and students working on STAMP last year. After several years of using VB enhanced Excel spreadsheets we decided that STAMP was the best alternative and had the benefit of an existing platform on which to build. At all 3 competitions we attended last year (Buckeye, Wisconsin, Champs) we set up our wireless network and let the data fly. At this point input into the system is only allowed for our team members (garbage in = garbage out) so we would be getting and giving accurate information. We did share any and all of the data with whoever wanted it and attracted a few interested individuals and teams in Atlanta. Our lead scouting mentor went so far as to put a match predictor into the program which accurately predicted our last three matches at the Wisconsin Regional. This sort of capability can be a real double-edged sword if you believe in self-fulfilling prophecies.

Bottom line is we like making our decisions based on cold, hard facts but when it comes down to it there is a certain "gut feeling" that can't be ignored. Instinct is one of those things you can't assign a number to.

Team 93 will remain a PC based scouting team. Faster, easier, more accurate, and a whole lot less paper to lug around!

My 2 pesos.

Sean

[Cory]

We used a system that sounds similar in principle to 25's.

Im not sure about any technical details, as that sort of thing isn't my specialty, but the kids that wrote the program did a fantastic job. It came in very handy for us throughout the season.

The only thing I didn't necessarily like about computer based scouting is that it becomes way too easy to just let the computer tell you that team xyz is statistically the best, and to go solely off the rankings. The scouting guys would tell us that the program says one team is the best bet for us to pick, when the mentors and drive team all thought that other robots fit our strategy better, among other things. You have to be careful to supplement your personal knowledge of teams with the computer, rather than rely on the computer.

[Bill Moore]

A quick search of last year's "At the Event" section has yielded no such rule. Does anyone know the rule number or what section I can find it in? I seem to remember people picking up on WiFi networks being run by teams last year at competitions (Nintendo DSs are excellent network sniffers.)
There was no rule banning wireless networks at competitions. This is just a FIRST Urban Legend. In fact, one team was actively trying to collaborate with other teams by sharing a wireless network for scouting.

We need someone to start "www.FIRSTsnopes.com" :D

[themagic8ball]

I didn't have time to read this whole thread but Team 537 used MARRS this year. It is a paper-enter-into-database method many teams used and I was responsible for writing the Database portion of it. I did it in PHP and MySQL and we had a very efficient system of multiple data-enterers and one printer to print out scouting reports (with pictures) of each team. It was invaluable during the course of competition to have up to the minute stats on each team. I will be once again writing MARRS this year and making it available for other teams to use.

[Graham Donaldson]

The only thing I didn't necessarily like about computer based scouting is that it becomes way too easy to just let the computer tell you that team xyz is statistically the best, and to go solely off the rankings. The scouting guys would tell us that the program says one team is the best bet for us to pick, when the mentors and drive team all thought that other robots fit our strategy better, among other things. You have to be careful to supplement your personal knowledge of teams with the computer, rather than rely on the computer.

That is why it is important to have an opinion box on a form where a scouter can put in an opinion thing, ex. where a robot did not get up on the ramp but didn't because they didn't have enough time or were blocking the other ramp. As you said, the problem with plain statistics is that they sometimes hide the real details.

Conclusion: statistics are great, but personal opinion is critical as well.

The Canuck

[Nitroxextreme]

That is why it is important to have an opinion box on a form where a scouter can put in an opinion thing, ex. where a robot did not get up on the ramp but didn't because they didn't have enough time or were blocking the other ramp. As you said, the problem with plain statistics is that they sometimes hide the real details.

Conclusion: statistics are great, but personal opinion is critical as well.

The Canuck

Couldn't be put better...

Opinion as well as talking with the team personally does wonders...
some team may have been trying the whole regional to do one thing and did that one thing OK...but in their last match they figure out that they need to be doing something completely different and then they just kick $@#$@#$@#

[Cory]

That is why it is important to have an opinion box on a form where a scouter can put in an opinion thing, ex. where a robot did not get up on the ramp but didn't because they didn't have enough time or were blocking the other ramp. As you said, the problem with plain statistics is that they sometimes hide the real details.

Conclusion: statistics are great, but personal opinion is critical as well.

The Canuck

Sometimes personal opinion is no good either.

One of our scouts said the team that played the best defense on us all year long played "poor defense", and they were watching the match it happened in :ahh:

[Graham Donaldson]

Sometimes personal opinion is no good either.

One of our scouts said the team that played the best defense on us all year long played "poor defense", and they were watching the match it happened in :ahh:

Yes, which is why statistics are also good.

An update to everyone monitoring this thread, or anyone interested in computer scouting in general: I am setting up a test Access database with last year's data from the Palmetto regional. When finished, I will see how I upload it (either here or as a white paper? I dunno), and you guys can take a look at it, and give feedback. Maybe use it as a base for yours for this year??? :D

The Canuck

[JamesBrown]

Well for any one who doesn't know I am in college now. I will be mentoring a team up here as well as stopping by 1568 a couple of time. However I wont be any where near as involved as I was the last 2 years. I have been the programmer for the last 2 years on 1568 however my interest is almost completely in scouting and strategy. I figure in my free time at school I can work on scouting. I am on the fence about whether I want to write a program from scratch to do scouting or if I want to use access. But in the end I will do one or the other.

If I was to come up with a good system that would scout all the criteria that effect this years game would other teams with less free time be interested? I would plan to make it very general so you could just input a team list for any regional and use it for that regional. If you would be interested would you find it fair that in exchange for using the program/database you would be asked to send me your scouting data for that regional so I could put it online somewhere where every one using the program/database could get that info and use it for their scouting at later regionals or championships, let me know what you think.

James

[Dylan Gramlich]

I believe that is a very good idea and i(personally maybe not my team but i will bring it up)would be very interested in this program. Thank you for volunteering your time for that

[Astronouth7303]

Is it possible to get data from the field systems somehow? It seems that a lot of tasks could be automated that way.

I know that it isn't currently possible, and that (to my knowledge) FIRST has not yet provided data. I also hear that someone else is replacing Hatch, so it may be possible to do this.

[FourPenguins]

Is it possible to get data from the field systems somehow? It seems that a lot of tasks could be automated that way.

I know that it isn't currently possible, and that (to my knowledge) FIRST has not yet provided data. I also hear that someone else is replacing Hatch, so it may be possible to do this.
Aww, but that'd take the fun out of it!

[Graham Donaldson]

Oh it would! And wouldn't give me anything to do at regionals, so I wouldn't go! That can't happen!

Ha ha.

The Canuck