New Global Scouting System

[singkiwi]

Over the past few days I have been developing a new type of scouting service, which will be different from what we have developed and implemented in the past. Right now it is in the early ideas and dev. stage, however I wanted to share what we are trying to do with it to get some interest going and hear everyone's opinions.

The main goal of the project is to be able to not only scrape data off of the FRC team database, but also use an email address to collect data submitted via email by other teams. The main idea is that we can take statistics emailed in from not only our team, but from other teams as well, and combine it all into a single online data sheet. (For example you could e-mail data, text it to the account, or any other way which best suits you). This data from the emails would be gathered and updated automatically, and placed online for everyone to see and review, allowing teams to spend less time gathering and sorting scouting data by having a large database of scouting data.

What does everyone think about this technique? Would you use it? What are some shortfalls with this system? What kind of capability would you like it to have?

Thanks,
Dan from First Robotics Team 2170

[XaulZan11]

Originally Posted by singkiwi

What are some shortfalls with this system?

The biggest one is getting teams to submit scouting information. There have been a lot of efforts to collect information directly from teams and I can't think of any that succeeded in getting a useful amount of participation. Other potential issues include trusting other team's scouting data (I know I would take all scouting data from unknown teams with a big grain of salt) and dealing with the differences in scouting statistics.

[JewishDan18]

I am actually going to do something extremely similar to this during winter break. My idea was to run something via appengine to make a web frontend that anyone could access (with login via google account), and allow them to pull either all data, or data from their team's scouts.

An important (and unfortunate) problem to keep in mind is that someone could flood it with bad data, or with false positive data about their team, or negative data about some other team that they dislike for whatever reason.

One interesting application of this will be to meta-scout. Knowing what a particular team is looking for based on what they are entering for data would be cool.

What we need is the Facebook of scouting. I don't mean that in a way that it's like Facebook, I mean it in a way like things on Facebook are. People who have a Facebook go on it a lot, and there are often enough people on it to get correct information. If there were a scouting database where everyone could not only upload data, but verify data, then we would have a working, successful scouter. The only problem, like stated before, is that it's hard to get everyone to add data, and some teams don't have the data some scouters require.

Inc conclusion, unless we get a god of scouting, the best way to scout is to do it yourself.

[singkiwi]

I agree, verifying data and getting people to use the system are both things which need to be addressed. What if we had teams register their email / input device to a whitelist? That way only verified accounts can submit data. Also, if you get a high volume of data you can identify outliers, which usually is incorrect data. Would setting a minimum amount of data input for each result make the scouting accurate? Are these the right kind of safeguards you would need to implement to get an accurate collection of data?

The only problem with this whole thing (and scouting apps in general), is the lack of people willing to write down the info for every single match of every single robot. There may be a few people, but enough for verification and an entire-team list would be quite difficult. Whitelisting it would only make there be fewer people.

[Tim Delles]

Originally Posted by SuperNerd256

The only problem with this whole thing (and scouting apps in general), is the lack of people willing to write down the info for every single match of every single robot. There may be a few people, but enough for verification and an entire-team list would be quite difficult. Whitelisting it would only make there be fewer people.

Let's be honest... the other question is how reliable the information is. You can have all of the information you want, but, if it's incorrect it is completely useless.

Hence why I wouldn't count on anything like this for scouting.

[slijin]

Originally Posted by Tim Delles

Let's be honest... the other question is how reliable the information is. You can have all of the information you want, but, if it's incorrect it is completely useless.

Hence why I wouldn't count on anything like this for scouting.

To add to that, it's all quantitative data, which discounts factors such as robot breakdown and technical malfunctions (not necessarily because of the team itself), as well as team v alliance performance. That isn't to say I don't like the idea - it sounds great, but for good scouting it seems relatively limited imho.

With regards to reliable data, perhaps you could have a verification system. The way I imagine it is that you'd have something like a "data queue", where all data first sent in is not immediately uploaded to the database, but to a tmp database, and then if two teams submit the same data for a match, the data is then uploaded.

[Q. Sheets]

What I've found in hosting a community scouting system is obtaining information on teams' robots and the general results of a match (alliance scores, bonuses, penalties and team DQs) is relatively easy.

Getting others to submit information on individual teams' performances during each match is where the issue lies. Every team quantifies the data differently. And then getting them to take the time to submit data for each robot in each match is a bother. I've tried.

For example, take the 2011 BAE Systems Granite State Regional. There are 87 qualifying matches which means 522 individual performances to record. And then with all the other events during the same weekend, the total is brought up to 503 qualifying matches (3,018 performances). Then someone/something has collate the inputted data and output it in some sort of structured format; this is a lot of work to do for something that may or may not be viewed depending partly on if the team attends another event that season… Unless you can get the turn-around rate high enough that it can be used during the same event, then more power to you.

I also cannot stress enough the fact that many teams still use pen(cil) and paper for scouting.

[singkiwi]

Thank you for all of your helpful responses! It has really helped me take this project to the next stage.

My current strategy based on all of your feedback:
1. Make a whitelist for 'trusted' scouters who are consistently reliable. At the same time, those who are 'untrusted' need to have their stats confirmed by another team's post or by an admin before their data is passed onto the main database.
2. Only whitelisted members can post their own teams stats (to prevent manipulation of the site... Even though this would never happen ).
3. Active Service: The program runs off site and is not reliant on user input
4. People submit information by sending an email / TEXT (That's the driving idea behind this: most people are able to quickly text in data. No internet or computer or fussy sheets needed!).
5. Data will be submitted in a specific format. This data is then sorted and used to calculate scouting data which can be found by sending a text to the email.
6. That leads into the best part: It's all interactive. Sending 'help' sends the format guidelines and a how-to guide. If it's your first data entry it will ask you to sign onto the service. If you format the message wrong, it will tell you.
7. The service will also incorporate the FRC alliance match scores and other FRC provided data.
(I'm also considering putting in a section about the status of team's robots, but I'm not willing to put in a comments section. Might do a sort of multiple choice question. I'll implement this much later.)

I'm also considering setting up a google docs page to display the final data spreadsheet to add some more advanced features.

How does all of this sound? If many people use the service, could it be effective? Am I missing a fatal flaw? Please let me know!

[Q. Sheets]

Originally Posted by singkiwi

4. People submit information by sending an email / TEXT (That's the driving idea behind this: most people are able to quickly text in data. No internet or computer or fussy sheets needed!).

By "TEXT," do you mean SMS messages?! If so, how do you plan on setting up such a system? Hopefully, you won't be spending real money on this before it's being used.

[singkiwi]

Originally Posted by Q. Sheets

By "TEXT," do you mean SMS messages?! If so, how do you plan on setting up such a system? Hopefully, you won't be spending real money on this before it's being used.

Yes, I do mean SMS Messages . Something not commonly known is that you can send text messages to email accounts for no additional charge, it just counts toward your text message count for the month (so if you have an unlimited text message plan any number you send / receive is free). And no, I'm not using money to make this possible. I am using an offline script I wrote to do all of the work, and am using a Gmail account to send / receive the messages.

[~Cory~]

May I suggest providing a base system that handles all of scouting for a team. If you are willing to make the system simple, fast, and accurate (Really high quality, enough for a team to want to pay money for it) that doesnt depend on other teams input, you can get somewhere. By default, include an option to send all collected data anonymously (some teams put very "personal comments" in their scouting reports... its not nice to find them on the ground ) to a centralized collection point. Teams who would want to utilize the global data could view the submitted sets on their own (but unlike as you said, make it so that the data is NOT combined all at once, instead make it so bias can be viewed easily and assessed).

1. Make a product that everyone wants (teams will be willing to help).
2. Submit that data collected (anonymously)
3. Allow teams to re download the data in a set-chunk form. Either info can be viewed by collection or by team, or by both