A New Way to Scout

[Andrew Schreiber]

Originally Posted by AdamHeard

We run a VERY large gdocs based scouting system, that is doing a lot more than just averaging (such that when data is entered, a lot recalculates).

Even with this, it runs. I suggest you look into what runs fast/slow on gdoc, and thoroughly understand the process it uses when to update cells. Short answer is anytime cell A changes, every cell that references it upates, and that cycle repeats.

Don't use vlookups. At all. Even once. Just don't do it. Use index(match()) instead. Google it for good tutorials, this doesn't load the range per each cell called. Importrange/sortrange can also be used where vlookup would be used by some people with better results. More or less, don't use vlookup and don't use cell by cell calls to pass entire ranges if you can help it.

If you want me to take a look at your system and offer more specific advice I can. I'm not saying this to criticize you guys, but I'm 100% sure the issue here is implementation, not google docs.

Not to criticize, honest question - If you're already doing VLOOKUP and INDEX type operations why not invest a little bit of time to learn how to do this stuff with a full blown relational database? It'd be faster performing, more scalable, more reliable, and teach students useful skills. I guess the only real issue would be a simple front end but that's not that bad to do any more…


Disclaimer - my bread and butter is databases, Postgres, SQL-Server, Mongo, CouchDB… When all you've got is a hammer every problem looks like a nail, I'm genuinely curious why so many people seem to shoehorn excel into a job for an actual DB.

[AdamHeard]

Originally Posted by Andrew Schreiber

Not to criticize, honest question - If you're already doing VLOOKUP and INDEX type operations why not invest a little bit of time to learn how to do this stuff with a full blown relational database? It'd be faster performing, more scalable, more reliable, and teach students useful skills. I guess the only real issue would be a simple front end but that's not that bad to do any more…


Disclaimer - my bread and butter is databases, Postgres, SQL-Server, Mongo, CouchDB… When all you've got is a hammer every problem looks like a nail, I'm genuinely curious why so many people seem to shoehorn excel into a job for an actual DB.

People/Resources. We can't dedicate programmers to this, and we're generally so short on programmers they are on the robot hte entire time.

Excel/Gdocs is a useful tool for ALL engineers, so in theory every student on the team should learn it.

A viable scouting system can be made in an hour easily if you're just doing averaging like most teams are. Less by an experienced user. You don't have to worry about potential errors/bugs you've introduced, etc... I don't buy the increased reliability argument.

Furthermore, the real big gain, is distribution. Every kid on the team has a google account, they have the scouting folder shared. Phones, tablets, laptops, etc... can all view it with no pain or hassle.

User interface as well, the UI is so much faster and easier to edit for people with no programming experience.

It has a lot of power for so little effort. It's the same reason why so many teams buy COTS shifters.

[Nemo]

(Damp Robot): I don't know if I would trust data that's gathered by "the community" for more than cursory overviews of robots. Our scouters all know the value of the data they collect, and see that it's of a real benefit to our team in qual matches and in alliance selection. That means that if they collect the data, it's good data. If an amorphous group of kids from different teams was working on the same data, they likely wouldn't have the same motivation to "get it right." You might end up with missing matches, people over counting their own teams, or just general slacking off. Of course, I agree having only one group of 7 per regional scout would be a ton less work, but potentially at the cost of data quality.

Originally Posted by Qbot2640

And to the "data-sharing" issue...I definitely believe you should have to contribute to benefit. From my perspective, this benefits the rookie teams or smaller teams much more. Consider, if there was no investment required to obtain this data, the teams with the best scouting programs could hold all their own scouters for their own data, yet still get the larger group data. A smaller or less experienced team might not be able to generate good data themselves. On the other hand, if you need to contribute to the effort to gain access to the data the former team WILL participate because they won't want to risk missing something that everyone else will have access to. Their benefit will still be considerably less than the latter team, since the data they would have without cooperation would already be pretty good.

If you want good data, you can't force people to scout. I think we all agree on this point. Where this discussion leads from here depends on the goals of a collaborative scouting project. If the goal is for a few teams to work together to build some connections and gain a collective competitive advantage over the rest of the teams, then it might make sense to only allow access to the data to teams who contributed. That's not really the way I think of this. I think everyone would be better off if the data was freely available to anyone who wants it.

The way to do this would be to incorporate the data collecting into the event volunteer staff. Teams provide the volunteers, and those people take the requisite training just like they would if they want to become inspectors / referees / etc. Then they collect quality data in their roles as the official scorekeepers, and stats get released for public consumption. Just like they do in athletics.

The lack of this type of official scoring data is a deficiency in the FRC program. It's not like FRC doesn't have enough on its plate for the few people who actually get paid to run the operation, so that's not a criticism of them. I just think this is something that the community and FRC should address in the long term.

If we want FIRST to be loud, having scoring statistics readily available would help a bit. It gives people something to look at online to learn who's who, and that draws certain types of people in. It would make it possible to produce events with commentary and some statistics incorporated into the broadcast to add context and history to the matches to make them more interesting to casual observers and newbies. Without that stuff, it's harder for an observer to know or care about the difference between teams 6875 and 6587 and 6758.

[Alpha Beta]

Originally Posted by Chadfrom308

Really? I have never experienced any lag or slowness from google drive, and it has been up 100% of the time...

Has anybody tried using IFTTT to text(SMS) data to a shared spreadsheet? Anyone with a cell phone could contribute to the data pool.

Scouters would need to open an IFTTT account and give it access to receive SMS/Texts (normal messaging rates apply), and Google account information to write to their Google Drive.

A Master Scout would need to designate a Spread Sheet file and share it with the other scouts. (All scouters would also automatically have access to the raw data in their Google Drive as well.)

If crowd scouting: On Thursday set up a scouting kiosk with a cellular hot spot for potential scouters to set up their account... or have the regional director e-mail all the teams ahead of time the set-up instructions.

PS. You can opt in the recipe to send/collect the phone number of the SMS sender. This would allow the master scout to text back anybody who is doing it wrong, or sort out all of their data and remove it from the pool. This would also release the phone numbers of the scouting participants to each other (no names attached). Not sure if that is a problem.

[jlmcmchl]

Originally Posted by Alpha Beta

Has anybody tried using IFTTT to text(SMS) data to a shared spreadsheet? Anyone with a cell phone could contribute to the data pool.

Scouters would need to open an IFTTT account and give it access to receive SMS/Texts (normal messaging rates apply), and Google account information to write to their Google Drive.

A Master Scout would need to designate a Spread Sheet file and share it with the other scouts. (All scouters would also automatically have access to the raw data in their Google Drive as well.)

If crowd scouting: On Thursday set up a scouting kiosk with a cellular hot spot for potential scouters to set up their account... or have the regional director e-mail all the teams ahead of time the set-up instructions.

PS. You can opt in the recipe to send/collect the phone number of the SMS sender. This would allow the master scout to text back anybody who is doing it wrong, or sort out all of their data and remove it from the pool. This would also release the phone numbers of the scouting participants to each other (no names attached). Not sure if that is a problem.

I use IFTTT for daily use, and I never realized it could do this. That's awesome.

I'm not going to grab a spreadsheet at the moment, but can you put data into hidden columns? Make the spreadsheet view only(And thus they can't directly open the hidden column) to everyone except the owner, and make a copy for the owner to check phone numbers in the hidden columns. From there, it's up to the owner to know when data is correct or faulty and let the users know how to improve their data input.

[Alpha Beta]

Originally Posted by jlmcmchl

I use IFTTT for daily use, and I never realized it could do this. That's awesome.

I'm not going to grab a spreadsheet at the moment, but can you put data into hidden columns? Make the spreadsheet view only(And thus they can't directly open the hidden column) to everyone except the owner, and make a copy for the owner to check phone numbers in the hidden columns. From there, it's up to the owner to know when data is correct or faulty and let the users know how to improve their data input.

As far as I can tell, people who have a shared copy of the document cannot enter data unless they have the rights to edit the spreadsheet, including every column that IFTTT is trying to write to. I did verify that they can enter data while the column is hidden.

Even if the owner hides and protectes a column the shared users could still make a copy of the spreadsheet and then as the owner of the copy unhide and unprotect the information.

[saikiranra]

Originally Posted by AdamHeard

We run a VERY large gdocs based scouting system, that is doing a lot more than just averaging (such that when data is entered, a lot recalculates).

Even with this, it runs. I suggest you look into what runs fast/slow on gdoc, and thoroughly understand the process it uses when to update cells. Short answer is anytime cell A changes, every cell that references it upates, and that cycle repeats.

Don't use vlookups. At all. Even once. Just don't do it. Use index(match()) instead. Google it for good tutorials, this doesn't load the range per each cell called. Importrange/sortrange can also be used where vlookup would be used by some people with better results. More or less, don't use vlookup and don't use cell by cell calls to pass entire ranges if you can help it.

If you want me to take a look at your system and offer more specific advice I can. I'm not saying this to criticize you guys, but I'm 100% sure the issue here is implementation, not google docs.

Sounds kind of like our old system.

That is probably it. We have a TON of vlookups, and that probably messed everything up. After finding the vlookups, we never tried to look for a more efficient method.
(Although I still am worried about losing the online data, after it consistently happened to us)


A few months ago, I would have loved for you to take a look at the system, but I invested quite literally my entire summer creating our own proprietary system, that somehow works.

Thank you for the offer!

[1306scouting]

I talked with the lead programmer for CrowdScout today, and got some more details confirmed:

We'll use a database backend (MySQL or similar) to store the data. We'll have an API so that we can take data from many sources over in a standardized format; we'll publish this API for anyone to use with their software, though we may need to set up API keys to prevent server overloading from unintentional traffic floods blocking it for others.

We should have no issues accommodating all the data from every match in every tournament, with one database per year. It turns out FRC actually creates a fairly small amount of data in a season, somewhere in the millions of data points range (Relative to some of the data sets I've worked with, at least.)

The API will just provide a way to upload data; every year we'll have a suggested list of metrics to score by, as well as taking other inputs (we're still rolling ideas around for the custom inputs, so that's not final.)

Data will be distributed to teams by both raw data access (bulk exporting to some storage file) and data processing (running weighting algorithms and so forth). We envision teams as doing their own visualizations/processing beyond that, so we're not going to have a massively complicated GUI with a million different buttons; we could never match the ingenuity of teams in how they want to do their own processing.

(Someone could easily create a frontend and make it public, but we think that's better done locally and that our time and energy is better spent on making a rock-solid reliable backend and great API.)

[Andrew Schreiber]

Originally Posted by 1306scouting

I talked with the lead programmer for CrowdScout today, and got some more details confirmed:

We'll use a database backend (MySQL or similar) to store the data. We'll have an API so that we can take data from many sources over in a standardized format; we'll publish this API for anyone to use with their software, though we may need to set up API keys to prevent server overloading from unintentional traffic floods blocking it for others.

We should have no issues accommodating all the data from every match in every tournament, with one database per year. It turns out FRC actually creates a fairly small amount of data in a season, somewhere in the millions of data points range (Relative to some of the data sets I've worked with, at least.)

The API will just provide a way to upload data; every year we'll have a suggested list of metrics to score by, as well as taking other inputs (we're still rolling ideas around for the custom inputs, so that's not final.)

Data will be distributed to teams by both raw data access (bulk exporting to some storage file) and data processing (running weighting algorithms and so forth). We envision teams as doing their own visualizations/processing beyond that, so we're not going to have a massively complicated GUI with a million different buttons; we could never match the ingenuity of teams in how they want to do their own processing.

(Someone could easily create a frontend and make it public, but we think that's better done locally and that our time and energy is better spent on making a rock-solid reliable backend and great API.)

Might I suggest taking a look at MongoDB. As it is schema less database you might have better luck, that way if teams don't collect some metric it won't be a complete mess for your DB and you don't have to have columns for everything. Simple clean JSON payloads

[khanh111]

Originally Posted by 1306scouting

I talked with the lead programmer for CrowdScout today, and got some more details confirmed:

We'll use a database backend (MySQL or similar) to store the data. We'll have an API so that we can take data from many sources over in a standardized format; we'll publish this API for anyone to use with their software, though we may need to set up API keys to prevent server overloading from unintentional traffic floods blocking it for others.

We should have no issues accommodating all the data from every match in every tournament, with one database per year. It turns out FRC actually creates a fairly small amount of data in a season, somewhere in the millions of data points range (Relative to some of the data sets I've worked with, at least.)

The API will just provide a way to upload data; every year we'll have a suggested list of metrics to score by, as well as taking other inputs (we're still rolling ideas around for the custom inputs, so that's not final.)

Data will be distributed to teams by both raw data access (bulk exporting to some storage file) and data processing (running weighting algorithms and so forth). We envision teams as doing their own visualizations/processing beyond that, so we're not going to have a massively complicated GUI with a million different buttons; we could never match the ingenuity of teams in how they want to do their own processing.

(Someone could easily create a frontend and make it public, but we think that's better done locally and that our time and energy is better spent on making a rock-solid reliable backend and great API.)

Our scouting software has been rebuilt annually since we started using software for scouting. This meant that one or two people spent over 300 collective hours during the season writing it. My plan for this year prioritizes maintenance, efficiency, and software design.

We are using:
node.js: Node is a tool for the modern web. It was created to be used for numerous client-server interactions in which small amounts of data were passed (think chatrooms). It is single-threaded, but extremely efficient. In all honesty, we could probably over a thousand FRC teams scouting concurrently with node (when used correctly). (Twitter did a study, and saw that using node for certain aspects of their system increased the amount of clients that a server could service from a few thousand to over a million).

express.js: This is a lightweight framework for node. Makes routing simple, among other small things.

mongodb w/ mongoose: mongo is the best option for use with node because it is nonblocking (unlike SQL). Both mongodb and node will be extremely efficient by servicing other requests while waiting for requests to be made and return. Mongoose is a nice wrapper library for mongo which makes it much easier, and adds schema support for mongo.

bootstrap: bootstrap is the most popular library on GitHub. We're using it as a frontend CSS framework to keep things simple.

angular.js: frontend JS framework made by Google. allows for two-way data binding, as well as easier maintenance of HTML. This is great, and eliminates some of the more boring code that deals with DOM manipulation.


In order to improve year on year, we have to use the most efficient system possible, while keeping it maintainable. I'm writing multiple wrapper APIs to further simply each of these parts of the frameworks.

Maybe we can work together to make a better overall crowdscouting system. I'd definitely want to see the details of how you currently do it.

[Mr. Mike]

Below is from last year before we went to wolds.

Originally Posted by iv597

We at 3138 wrote an HTML5 webapp, the "Innovators Predictive Analytics Scouting System" (IPA). This is designed to go beyond the usual "picklist" system many teams use their scouting data for, and into per-match strategy. Our, and other teams', drive team[s] were able to not only change personal strategy to be more effective for the upcoming matches, but collaborate with the alliance to sometimes change the predicted outcome of the matches entirely. The data is also open to anyone with the URL, so it's not limited to helping our team and alliances: folks were dropping by our pit constantly at Worlds to check on their stats.

I am the client-side developer of this system and would be more than willing to explain this in more detail if needed. I could also work with you to set it up for your event, if you wished. (As a matter of fact, we're already using it for an event 3138 is not competing in: we're sending some folks over to IRI this week to use the system and demo for a few teams that may be interested in seeing it in use).

For now, though, feel free to read the thread here:

http://www.chiefdelphi.com/forums/sh...d.php?t=116017

[1306scouting]

Khanh111: I've PMed you about a collaboration.

IPA is really cool, and I'd love to integrate it if 3138 is willing; not all teams would be able to submit all the data that IPA uses (paper based teams couldn't submit shot location, for instance), but we could definitely store that data on the platform we're developing, and the client would have access to both the specific data it needs as well as access the rest of the data.

[yash101]

With HTML5, you get drag n drop! So, record the position of robot shoot point(s) and store them in a database.

After thinking for a lot of time, I was thinking about which Database I should use. My hosting platform is a Raspberry Pi, so I do not have too much power. What database should I use for such an intensive application?

Single MySQL Database
Multiple MySQL Databases
Single Text File
Multiple Text Files
Something else?

[1306scouting]

Originally Posted by yash101

With HTML5, you get drag n drop! So, record the position of robot shoot point(s) and store them in a database.

After thinking for a lot of time, I was thinking about which Database I should use. My hosting platform is a Raspberry Pi, so I do not have too much power. What database should I use for such an intensive application?

The issue is for teams like mine that scout on paper sheets - there's no reasonable way to encode where the shots are coming from on a piece of paper (though I think OMAR, our scan-reading software, could probably do it if we wanted it to).

(Warning: speculation ahead, I could be wrong about this: )

Honestly, a Raspberry Pi doesn't have nearly enough power to run this scale an application. You need a lot more RAM for DBs; the 512MB on the Raspberry Pi probably won't be enough, I'd want at least 2-4GB for this size of database with the amount of data that we could be working with. Part of that is me wanting plenty extra for comfort, since the database may be able to run in 512MB, but part is the fact that I use more than 512MB while my web servers are idle on my personal server (admittedly not heavily optimized, but also not nearly as intensive as the sort of database we'll be using.)

[yash101]

Maaannn!!! What do you run on that poor server? My server uses just 192MB on standby, running all the crapware I keep on it!
See the screenshot. BTW, I was using SSH Display forwarding, so I was hogging up CPU resources and RAM by that. I do not keep LXDE running.

Lol. I was writing this post while taking the screenshot!