I've tried many different methods of scouting over the years with the various teams I've been on, and I've also had the opportunity to observe how 1114 does it. Based on that, I've converged on this list of requirements for a scouting system that approaches the optimal:
1) No reliance on an internet connection -- too many venues don't offer it, and mobile hotspots can be flaky
2) Data collection uptime must be 100% -- you only get one opportunity to collect each piece of data from a match
- Corollary: data collection can't depend on devices that require power
- Corollary: data collection needs to be easy -- it can be boring and we want to facilitate rotation between any of the students sitting in the stands
3) Data entry can lag collection, but we want aggregate scouting data to be available throughout the day, to help with qualification match strategy
4) Data entry needs to be 100% accurate
Cheesy Scout satisfies (3) and (4), but falls short on (1) and (2). 1114's system (paper, with manual data entry completed in time for the picklist meeting) satisfies (1), (2) and (4), but doesn't provide (3). Last year, we collected the data on paper and then a couple of students entered it into the computer as fast as they could, which satisfied (1) and (2) but was slightly too slow for (3) and had some issues with (4), not to mention that data entry was a tough and undesirable job.
So this year our paper scouting sheets will have bubbles to fill in and we'll task someone to feeding them into the scanner. If we lose power to the scanner/computer in the stands we can easily relocate it somewhere else without losing data.
1503 did
something like this in 2011, and it was pretty effective although there were issues with the scanner used. I'm hoping to fix that this year with a higher-quality scanner.
Of course, this only represents about half of our entire scouting plan. Come find me at competition sometime and I'll be happy to talk you through the pit scouting, data analysis and picklist meeting parts as well.