A New Way to Scout

[Andrew Schreiber]

Originally Posted by AdamHeard

We run a VERY large gdocs based scouting system, that is doing a lot more than just averaging (such that when data is entered, a lot recalculates).

Even with this, it runs. I suggest you look into what runs fast/slow on gdoc, and thoroughly understand the process it uses when to update cells. Short answer is anytime cell A changes, every cell that references it upates, and that cycle repeats.

Don't use vlookups. At all. Even once. Just don't do it. Use index(match()) instead. Google it for good tutorials, this doesn't load the range per each cell called. Importrange/sortrange can also be used where vlookup would be used by some people with better results. More or less, don't use vlookup and don't use cell by cell calls to pass entire ranges if you can help it.

If you want me to take a look at your system and offer more specific advice I can. I'm not saying this to criticize you guys, but I'm 100% sure the issue here is implementation, not google docs.

Not to criticize, honest question - If you're already doing VLOOKUP and INDEX type operations why not invest a little bit of time to learn how to do this stuff with a full blown relational database? It'd be faster performing, more scalable, more reliable, and teach students useful skills. I guess the only real issue would be a simple front end but that's not that bad to do any more…


Disclaimer - my bread and butter is databases, Postgres, SQL-Server, Mongo, CouchDB… When all you've got is a hammer every problem looks like a nail, I'm genuinely curious why so many people seem to shoehorn excel into a job for an actual DB.

[AdamHeard]

Originally Posted by Andrew Schreiber

Not to criticize, honest question - If you're already doing VLOOKUP and INDEX type operations why not invest a little bit of time to learn how to do this stuff with a full blown relational database? It'd be faster performing, more scalable, more reliable, and teach students useful skills. I guess the only real issue would be a simple front end but that's not that bad to do any more…


Disclaimer - my bread and butter is databases, Postgres, SQL-Server, Mongo, CouchDB… When all you've got is a hammer every problem looks like a nail, I'm genuinely curious why so many people seem to shoehorn excel into a job for an actual DB.

People/Resources. We can't dedicate programmers to this, and we're generally so short on programmers they are on the robot hte entire time.

Excel/Gdocs is a useful tool for ALL engineers, so in theory every student on the team should learn it.

A viable scouting system can be made in an hour easily if you're just doing averaging like most teams are. Less by an experienced user. You don't have to worry about potential errors/bugs you've introduced, etc... I don't buy the increased reliability argument.

Furthermore, the real big gain, is distribution. Every kid on the team has a google account, they have the scouting folder shared. Phones, tablets, laptops, etc... can all view it with no pain or hassle.

User interface as well, the UI is so much faster and easier to edit for people with no programming experience.

It has a lot of power for so little effort. It's the same reason why so many teams buy COTS shifters.

[saikiranra]

Originally Posted by AdamHeard

We run a VERY large gdocs based scouting system, that is doing a lot more than just averaging (such that when data is entered, a lot recalculates).

Even with this, it runs. I suggest you look into what runs fast/slow on gdoc, and thoroughly understand the process it uses when to update cells. Short answer is anytime cell A changes, every cell that references it upates, and that cycle repeats.

Don't use vlookups. At all. Even once. Just don't do it. Use index(match()) instead. Google it for good tutorials, this doesn't load the range per each cell called. Importrange/sortrange can also be used where vlookup would be used by some people with better results. More or less, don't use vlookup and don't use cell by cell calls to pass entire ranges if you can help it.

If you want me to take a look at your system and offer more specific advice I can. I'm not saying this to criticize you guys, but I'm 100% sure the issue here is implementation, not google docs.

Sounds kind of like our old system.

That is probably it. We have a TON of vlookups, and that probably messed everything up. After finding the vlookups, we never tried to look for a more efficient method.
(Although I still am worried about losing the online data, after it consistently happened to us)


A few months ago, I would have loved for you to take a look at the system, but I invested quite literally my entire summer creating our own proprietary system, that somehow works.

Thank you for the offer!

[Alpha Beta]

Originally Posted by Chadfrom308

Really? I have never experienced any lag or slowness from google drive, and it has been up 100% of the time...

Has anybody tried using IFTTT to text(SMS) data to a shared spreadsheet? Anyone with a cell phone could contribute to the data pool.

Scouters would need to open an IFTTT account and give it access to receive SMS/Texts (normal messaging rates apply), and Google account information to write to their Google Drive.

A Master Scout would need to designate a Spread Sheet file and share it with the other scouts. (All scouters would also automatically have access to the raw data in their Google Drive as well.)

If crowd scouting: On Thursday set up a scouting kiosk with a cellular hot spot for potential scouters to set up their account... or have the regional director e-mail all the teams ahead of time the set-up instructions.

PS. You can opt in the recipe to send/collect the phone number of the SMS sender. This would allow the master scout to text back anybody who is doing it wrong, or sort out all of their data and remove it from the pool. This would also release the phone numbers of the scouting participants to each other (no names attached). Not sure if that is a problem.

[jlmcmchl]

Originally Posted by Alpha Beta

Has anybody tried using IFTTT to text(SMS) data to a shared spreadsheet? Anyone with a cell phone could contribute to the data pool.

Scouters would need to open an IFTTT account and give it access to receive SMS/Texts (normal messaging rates apply), and Google account information to write to their Google Drive.

A Master Scout would need to designate a Spread Sheet file and share it with the other scouts. (All scouters would also automatically have access to the raw data in their Google Drive as well.)

If crowd scouting: On Thursday set up a scouting kiosk with a cellular hot spot for potential scouters to set up their account... or have the regional director e-mail all the teams ahead of time the set-up instructions.

PS. You can opt in the recipe to send/collect the phone number of the SMS sender. This would allow the master scout to text back anybody who is doing it wrong, or sort out all of their data and remove it from the pool. This would also release the phone numbers of the scouting participants to each other (no names attached). Not sure if that is a problem.

I use IFTTT for daily use, and I never realized it could do this. That's awesome.

I'm not going to grab a spreadsheet at the moment, but can you put data into hidden columns? Make the spreadsheet view only(And thus they can't directly open the hidden column) to everyone except the owner, and make a copy for the owner to check phone numbers in the hidden columns. From there, it's up to the owner to know when data is correct or faulty and let the users know how to improve their data input.

[Alpha Beta]

Originally Posted by jlmcmchl

I use IFTTT for daily use, and I never realized it could do this. That's awesome.

I'm not going to grab a spreadsheet at the moment, but can you put data into hidden columns? Make the spreadsheet view only(And thus they can't directly open the hidden column) to everyone except the owner, and make a copy for the owner to check phone numbers in the hidden columns. From there, it's up to the owner to know when data is correct or faulty and let the users know how to improve their data input.

As far as I can tell, people who have a shared copy of the document cannot enter data unless they have the rights to edit the spreadsheet, including every column that IFTTT is trying to write to. I did verify that they can enter data while the column is hidden.

Even if the owner hides and protectes a column the shared users could still make a copy of the spreadsheet and then as the owner of the copy unhide and unprotect the information.

[khanh111]

Originally Posted by Roger

There is another method, which, for lack of a better term, I would refer as the "Mycroft Holmes" method (Wikipedia's description is incomplete). It is someone that somehow can keep statistics of all the teams in his/her head, and be able to balance off which team would be good, and which to watch out. We've had a couple like that, and they are much better than any other system. I told the team's teachers to look out for a sports fanatic student for this role.

Our head scout for the last few years was this type of person. His skills were all enhanced by our data collection. It also made it easier for him to explain reasons to the rest of the team, which in turn allowed him to better develop strategies.

If anything, I think the "Mycroft Holmes" method should be supplemented by some form of collected hard data.

[XaulZan11]

Originally Posted by khanh111

If anything, I think the "Mycroft Holmes" method should be supplemented by some form of collected hard data.

I agree 100%. There is no substitute for hard data, but having someone that actually remembers what the robots look like, how they play,what they do in a match and so on is very benefical. At events for my team, I watch one or two teams a match and write qualitative notes down. At the end of the event I know the basics of every team (and a lot about certain teams) and could make a pretty good pick list. But, I'm always surprised when going through the data to see how well or how poorly some teams actually do (consistency, especially in autonomous, is typically the biggest source for deviation). It simply comes down to not being able to watch every match for every team.

[brandon.cottrell]

Originally Posted by Roger

There is another method, which, for lack of a better term, I would refer as the "Mycroft Holmes" method (Wikipedia's description is incomplete). It is someone that somehow can keep statistics of all the teams in his/her head, and be able to balance off which team would be good, and which to watch out. We've had a couple like that, and they are much better than any other system. I told the team's teachers to look out for a sports fanatic student for this role.

While our team does have it's own form of scouting, I kinda think that this method is what suits me the most. I like to look at regionals I didn't attend and try to work out what teams had the most synergy, and what the best possible alliances would be.

[mechanical_robot]

Originally Posted by khanh111

Crowdscouting would be a way for multiple teams that go to different events to contribute to the same dataset, which would be accessible by all members of the crowdscouting alliance (and publicly released after the season).

I like your idea and I think it would work but I don't see the reason to keep the data private during the competition season. There is no hurt that I know of from the data being public. Also that would help the rookie teams or teams that don't have a proper scouting team in place to collect data. Also what is the point of the data after the season, it is more like a incentive that isn't really an incentive to the teams that aren't part of this but want the data or need the data. I don't know. Just seems improper that you aren't releasing the data to the public and keeping it private until the season is over.

[khanh111]

Originally Posted by antimatter_john

I like your idea and I think it would work but I don't see the reason to keep the data private during the competition season. There is no hurt that I know of from the data being public. Also that would help the rookie teams or teams that don't have a proper scouting team in place to collect data. Also what is the point of the data after the season, it is more like a incentive that isn't really an incentive to the teams that aren't part of this but want the data or need the data. I don't know. Just seems improper that you aren't releasing the data to the public and keeping it private until the season is over.

The reason I would make the data private during the season is because having data is currently a major competitive factor. Until we reach a point where there's some third party scout that makes all data public, data collection will be a huge playing card at any event.

To clarify, "private" as I used it meant accessible to all teams who contributed to the dataset.

There will be some point at which scouting data and stats will become widespread, and at that point, the challenge will change primarily to using the data effectively. (think stock market - there's a lot of data, and lots of opinions).

[mechanical_robot]

What advantage does it have over wether your teams lose or win though? Big teams already have their own scouting system and so does most medium sized teams. Also you could be helping rookie or new or small teams. I don't see the point of it being secret, it's not like it is a special robot strategy or robot function. It is merely harmless data.

[Kimmeh]

Originally Posted by antimatter_john

What advantage does it have over wether your teams lose or win though? Big teams already have their own scouting system and so does most medium sized teams. Also you could be helping rookie or new or small teams. I don't see the point of it being secret, it's not like it is a special robot strategy or robot function. It is merely harmless data.

Exactly. What makes scouting actually mean something is what you do with the data. You're sharing the raw data, not how it's manipulated to be effective for your team.

[Connerd]

One of the ways we've debated doing scouting this season, as opposed to the hundreds of sheets of paper, is having a webpage which all of the scouters can connect to on their laptops, phones, tablets, etc. through a dedicated mobile hotspot. This data is collected, and organized on the scouting lead's Ipad, which allows for medium speed data review, depending on the internet speed.
We haven't thought about it fully, we've just been talking about it. The scanner method might work better though, as it seems simpler and more cost-efficient.

[Jim Wilks]

Originally Posted by Connerd

One of the ways we've debated doing scouting this season, as opposed to the hundreds of sheets of paper, is having a webpage which all of the scouters can connect to on their laptops, phones, tablets, etc. through a dedicated mobile hotspot. This data is collected, and organized on the scouting lead's Ipad, which allows for medium speed data review, depending on the internet speed.

Check rule T04:

"Teams are not allowed to set up their own 802.11a/b/g/n/ac (2.4GHz or 5GHz) wireless communication (e.g. access points or ad-hoc networks) in the venue.

A wireless hot spot created by a cellular device would be considered an access point."